6.9 C
New York
Sunday, December 4, 2022

Python is Getting Faster. How a Team at Microsoft is Helping – Slashdot

Slashdot is powered by your submissions, so send in your scoop


And they’ve already begun working on features for future versions of Python. “You can also find out more about what the Faster CPython Team has in mind for 3.12 in their ideas repo on Github.”


The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
You trust MS now? They arguably suck less than Oracle, but that’s not saying much.
Microsoft does stuff to benefit … microsoft.
So you really have to ask yourself what’s in it for them. Sometimes what benefits them may not benenfit the rest of us.
But i’m definitely in the WTF ? camp on this. I have no idea why M$ would be doing anything for Python, much less trying to make it faster.
So I’ll go ahead and be suspicious as to why they are doing this. It has to benefit them, or they wouldn’t, and I just don’t see how this is useful for them.
Who knows, maybe some manager at microsoft will
Microsoft uses a LOT of Python internally.
Any improvement in performance results in millions in cost savings.
Not just internally – MS are one of the bigger cloud providers with their Azure platform.
Making Python faster for users outside Microsoft does not save Microsoft money. Even if they are using it on Azure, making Python faster would mean customers using less compute and paying Microsoft less money. On the other hand if Python is faster Microsoft saves money because it uses it internally.

Even if they are using it on Azure, making Python faster would mean customers using less compute and paying Microsoft less money.

Even if they are using it on Azure, making Python faster would mean customers using less compute and paying Microsoft less money.
Not necessarily: if Python uses less compute, then potential customer projects that were previously not cost-effective (either directly because of the compute cost, or indirectly because they’d need to be written in a “harder” language than Python, costing more development time, in order to be efficient enough) can suddenly become viable, meaning customers spend more money with Microsoft.
To put it another way, making Python more efficient means that each “unit of CPU time” can do more useful work, and hence

or indirectly because they’d need to be written in a “harder” language than Python, costing more development time, in order to be efficient enough

or indirectly because they’d need to be written in a “harder” language than Python, costing more development time, in order to be efficient enough
Python has the advantage of being also one of the more Windows-friendly languages out-there. (e.g. most of the pypi dependencies install nicely on Windows, most of the dependencies management work nicely there too, python core libraries have enough abstration to run reliably on things which aren’t POSIX, etc.)
Meaning that people running business app in Python on Azure are more likely to be developing it on a Windows Laptop.
Thus licence money for Microsoft for Windows, and any other Microsoft ecosystem they
It’s a fair point that if you can’t see the benefit for Microsoft, you’d need someone to fill in the blank because they aren’t going to be utterly altruistic. People have stepped up to offer up why Microsoft interests are what they are and how they can be compatible with reasonable benefit for all to fill in that gap. So his gut reaction to be wary when he can’t think of the Microsoft benefit is fine, so long as the responses are internalized.

So you really have to ask yourself what’s in it for them.

So you really have to ask yourself what’s in it for them.
Microsoft makes stupid amounts of money using open source both internally and in their product offering. Literally half of their largest revenue stream is selling people Linux in the Azure cloud.
When you have a WTF moment, remember that it’s very easy to build on and sell someone’s work than it is to re-invent your own. Even if the company isn’t outwardly open source friendly.
Thanks! Now I understand what’s in it for them.
I’m surprised they would use Python internally instead of C# or whatever it is we’re all supposed to be used now.

instead of C#

instead of C#
Just my guess: C# (and a lot of other stuff) was developed in an era of “first, make sure it helps us sell Windows.” Insiders know this and steer clear of the built-in booby traps. Python started out being O/S agnostic.
The jury is still out on whether any Microsoft-specific hooks got slipped into the new Python, intentionally or otherwise.

I’m surprised they would use Python internally instead of C# or whatever it is we’re all supposed to be used now.

I’m surprised they would use Python internally instead of C# or whatever it is we’re all supposed to be used now.
I didn’t know anyone selects a programming language for “internally” vs “externally”. I always thought programmers select a language based on what it offers to solve the problem and conditions at hand. I learn something new every day.
If Microsoft is spending any appreciable amount of money for Microsoft on the project, then it is a bit confusing. What do they care if python is slow or not? In fact, isn’t it a benefit if all this scientific computing which goes on is slow AF, since that sells more cloudy time?
But if they have only a few engineers on the project, then it’s not a big surprise. Maybe Microsoft is beginning to see the OSS light, not in terms of benevolence but benefit. I wouldn’t trust them not to practice further skulldugge
So you really have to ask yourself what’s in it for them. Sometimes what benefits them may not benenfit the rest of us.
You mean besides making Azure more attractive? Why else would they invest in non-MS platforms and open source tech? Oh right… it’s to make Azure more competitive with AWS.
A lot of Azure has Python examples. And a lot of python is run in Azure (especially there machine learning and AI services.) So it is in their own best interest to make python faster.
Did I make it in before all the dreary tech dinosaurs chime in with “embraceextend exteenguissssh!!!
Oh shit you beat me! I was going to complain about Micro$oft or Micros~1 depending on my mood.
Anyhow
You didn’t however make it in before the dreary tech dinosaurs mindlessly ragging on python because python si teh sux etc. It’s almost like people here have endless hours to craft C code full of memory errors and don’t have a product to ship.
 
All languages suck, so that’s tautologically true.
Also, python isn’t my favourite language. I don’t love it but it’s fine. I’d certainly rather program in Python than not program at all.
You know what they say about ignoring the lessons of history? Trusting those proven to be untrustworthy is generally a mistake.
While I like the joke as FP, the discussion seemed to completely miss the point. At least I couldn’t find the obvious explanation of what is going on here…
A focus on speed means cutting out the glue code that makes things more flexible but slower. Alternatively, you could describe this as Microsoft’s ongoing attempt to glue “fast” Python to Microsoft’s other software, especially to the Windows OS.
So when are they getting rid of GIL? Still find it hard to believe people actually try to justify this crap.
Just use Jython on JVM, no GIL, and all the fun of running a bloated JVM

running a bloated JVM

running a bloated JVM
GraalVM fixes that. Seriously.
It’s weird making stuff in Java with memory usage that looks like Go.
Five years [backblaze.com], estimated. (Scroll to the bottom.)
What’s your problem with GIL? Python has dead simple multiprocessing and IO is multithreaded. I don’t see much value in full end2end multithreading support.
So far we given up too much single-threaded performance for multithreaded features. The later of which is a niche use case. If/When nogil comes online, I fully expect it to have a on/off flag.
I would much rather efforts be spent on Pipe, Queue, SharedMemory performance for multiprocessing. Or maybe add other useful classes.
IO is not all there is to life, and multiprocessing is dead simple only until it isn’t.
I’m doing some computations on large numerical arrays. In a sane world I would just write result = threaded_map(kernel, data), In Python I cannot do that because its authors could not be bothered to design their interpreter properly. Multiprocessing is not dead simple here, because ‘kernel’ can be a method of some object with nontrivial state that cannot be easily pickled (nor would you want to, because the state can be h
It’s not an excuse. You are dismissing 30 years of review on the topic as if all those people, many PhDs in Computing, are fools. Sorry if we take your bitter opinion with a grain of salt.
Your use case is really small… almost insignificant in the overall scheme. I doubt anyone is even working on a Python solution to your problem. And that’s if I assume your use case isn’t performant enough for SharedMemory or Pipes but still is for using an interpreted language.
And yes, compared to most other languages
Hurray! A faster way to justify getting all that awful Python code into prod (and watch it blow up)!
We’ve all certainly learned a lot from the ye olde days, and Python has a lot of modern aspects that promote dev happiness, but it’s still just another interpreted language. My team has to run 2 to 4 different linters against their Python source in CI, hoping to ensure it won’t accidentally blow up once deployed, and it takes far longer to certify syntactically than any compiled language I’ve ever used. The Go compiler absolutely destroys Python linters in terms time to complete “compiling”. Also, our Go CLIs have a much better user experience compared to the same functions in Python, in terms of execution speed.
Python does still have some utility, but I do think its been banging against the doors of maximum-effective-use-case for a while now. At least where I’m at, I feel like Python has been carrying more water than it should, and I think we should be spending our time on improving Go rather than shoring up Python.
You’re confusing interpreted languages with dynamic languages. CPython has always compiled down to an IL at run-time. Its slowness comes from its dynamic nature.
Python is just fugly. White space significance sucks. Just uses f’ing braces like any half decent language. Not to mention the syntax, and function names, are like something from VB 5.0.
I have to deal with Python on a regular basis and it’s just plain ugly. Horrible abbreviated, inconsistent funciton names just encourages people to start using horrible abbreviated variable names and the whole thing becomes barely legible crap.
Yuck…
Compared to the so called “one true brace style” the python solution seems almost elegant. Allman got it right.
You’re confusing the facilities offered by a language with the way people use it. Significant whitespace is an abomination and leads to context errors. Some brace styles might be ugly but having visible context block demarcators whether braces or words is a Very Good Idea which is why almost every mid and high level programming language ever invented uses them.
“Only thing that braces add is to make it possible to have misleading indentation without compile errors”
Better misleading indentation than bad logic. You can delete whitespace in python by mistake which moves the line into the outer scope and python will not say a word about it – your code just won’t work and good luck finding the bug. Now try deleting a brace in a C program and see how well it compiles.

You can delete whitespace in python by mistake which moves the line into the outer scope and python will not say a word about it – your code just won’t work and good luck finding the bug. Now try deleting a brace in a C program and see how well it compiles.

You can delete whitespace in python by mistake which moves the line into the outer scope and python will not say a word about it – your code just won’t work and good luck finding the bug. Now try deleting a brace in a C program and see how well it compiles.
I don’t see much difference. Most likely in python such an error would be easily detectable because different scopes have different local variables. And in C it would cause really long error cascade because missing brace would throw off the parser real good. Not much of a problem though if you know to check errors from
Yup I used to joke the best way to ruin a python program was to quickly do a select-all, dedent,dedent,dedent, save. Then lose undo history.
Any real language can survive that.
> Most coding conventions make you indent code properly anyway.
That depends on the language. By convention some (most?) people DON’T (heavily) indent the preprocessor in C/C++. This “TWO COLUMN” mode helps make debug-only code MORE readable since you decide which “column” to focus on when reading:
e.g.
. . . . foo();
#if DEBUG
. bar();
#endif
. . . . qux();

> You could as well consider proper indentation mandatory at which point having both indentation and braces/marker keywords becomes a violation of DRY principle.
No, for three reasons:
1. Define “proper” indentation because there are always edge cases where people will argue over what is “proper”.
2. There are many operations which are “dual” operations. Push/Pop, Startup/Shutdown, Open/Close. Indentation helps provide visual flow control especially when there are nested operations. Python’s retarded indentation dogma destroys this communication:
// Idiotic /. bug unindents
    pushMatrix();
        scale( foo );
        pushMatrix();
            rotate( bar );
        popMatrix();
    popMatrix();

Pythons pretends everyone is an idiot and makes them work harder due to some bullshit dogma:
<ecode>
// Idiotic /. bug unindents
    pushMatrix();
    scale( foo );
    pushMatrix();
    rotate( bar );
    popMatrix();
    popMatrix();

There ARE times to have braces, and there are times NOT to have braces. There ARE times to indent, there are times NOT to indent, and there are times to EXTRA indent. An expert knows when to follow and when to break Rules-of-Thumb.
3. Extra whitespace may help with readability to see the “flow” of variables. Consider this swap (ignore the inefficient ctor for tmp):
i.e.
template <typename T>
void swap( T &lhs, T &rhs )
{
    T tmp = lhs;
            lhs = rhs;
                  rhs = tmp;
}

Making whitespace change semantic meaning is absolutely retarded. We write code for people, not compilers. To every rule there is almost always an exception.
Your “dual operation” code could be nicely implemented as a context manager in Python, which would give you back your block indent of a group of operations, and even save you a line of code potentially, as the “close” part of the operation is handled in the context manager.
I used to hate white space significance. Then I kind of realized the white space it insisted on is exactly what I do anyway. It different, that doesn’t make it bad. It does make it easier to learn.
Python is very good for stuff where performance isn’t all that important. A great example is Ansible. There would be no gain whatsoever from converting that to Go or some other compiled language. It being Python has the added advantage that it’s easy for people to add their own modules to it.
Ansible is maddeningly, frustratingly slow. Some of that slowness may be intrinsically necessary due to what it does, but I’m surprised gathering facts takes more than a second. But everything is slow. (I use it anyway when I can. It’s worth the frustration.)
That’s basically because the methodology tends to be a slow way of doing things. It’s spending most of it’s time twiddling it’s thumbs compute wise. Just a lot of waiting and serialized execution of commands that in turn spend most of their time waiting. So a hypothetical C implementation of the exact same design would still be maddeningly slow. One difference is that a python developer *might* be more intimidated by C interfaces (they can use through one of several FFI facilities in python) and resort
Does speed really matter with Ansible? You start it off and go do something else while it works. It’s not an interactive process.
Python should have been somewhere in between a shell script and JSON, and used solely to tie disparate things together. Because of its poor performance it’s not even great at that job (perl is approximately four times faster at mangling data, which is something that you often want to do when connecting programs together) but its pretty indentation makes it a close cousin. Except, you know, JSON is basically comma-structured and can be regenerated on that basis, unlike Python 🙂
Instead people have developed
Sounds like you’re lacking good system testing before pushing to production. And complaining that the issue they’re fixing (performance) is a reason not to fix it.
I’m happy you like Go. I’ve heard good things about it, but there is a gravity/momentum to python that Go hasn’t overcome. Maybe you can identify it and help change things? I don’t think complaining that people use and work on python is the solve here.
I mean people love Rust too. And I’m sure many other languages. People love what they’re go
Before Py# is released as a new Microsoft language?
They already did. It’s called IronPython.
https://ironpython.net/ [ironpython.net]
They just brought in Jim Hugunin who wrote Jython. This was in 2006.
Sun knew why they didn’t make int an object…
According to this guys tests NON optimised C code ran 45 times faster than the same code in python. Optimised? Well you check it out:
https://peter-jp-xie.medium.co… [medium.com]
Sure, it’ll vary depending on algorithm but others have similar results and Daves Garage channel on youtube did a whole series on comparing language speeds and python didn’t do too well overall compared to pretty much anything.
And if you knew anything about Python, you’d know that for use cases where speed is the primary factor, you can stub out to C and back. That’s exactly how some of the more powerful libraries run – I’ve done a lot of code for the Python Imaging Library for example where that’s done, including true multi-threading support.
So – bash all you want, it just shows you don’t know when to use a hammer versus a screwdriver.
If you’re going to stub out to C you might as well write the whole lot in C or better yet C++ in the first place unless your clientelle is halfwit script kiddies who wouldn’t know one end of a pointer from the other.
Sure.
And you can reinvent hashtables, rudimentary string handling and an object lifetime strategy in C for the 100,000,000th time while you’re at it. It’s so fun!
Then you can buy a bunch of expensive 3rd party tools to try to identify all of the security vulnerabilities you just authored. With all the performance you’ll gain running one-time setup code 50X faster, surely it will pay for itself!
“And you can reinvent hashtables, rudimentary string handling and an object lifetime strategy in C for the 100,000,000th time while you’re at it. It’s so fun!”
Which is why I said C++. Though FWIW C has the hsearch library for decades.
“Then you can buy a bunch of expensive 3rd party tools to try to identify all of the security vulnerabilities you just authored”
Sure, or alternatively use the ones that come for free either with Visual Studio or on the linux command line.
I’m guessing you’ve never coded in eithe
I have decades of experience with C and C++.
Familiarity breeds contempt, especially with crap languages from 1990.
That’s a horrible comparison! I mean, I don’t like python myself, and it is definitely slow, but he compares the runtime of a compiled C program with the time to launch the python interpreter, compile and run the python code!
And the test code is so trivial, of course the optimization makes it a thousand times faster – if a compiler is smart enough it would do whatever sane human would do and assign the sum directly instead of looping!
Benchmarking is a tricky business, but this is retarted!
Make it faster at sucking!
Distinguished? Engineer? What makes this guy an engineer at all? And distinguished mostly in infamy for some of the worst design decisions and poorest performance ever.
They should say what they really did…surrounded to controller of Python with a few competent people so that the language could stop sucking after decades.
I am surprised to see no mention of PyPy [pypy.org]. I would have thought that if one wanted faster Python, that would be the place to start. Has there been any mention of why they didn’t go that way, what similarities there are, and where they differ?
GvR is embarrassed by the fact that he doesn’t understand PyPy, and at this point, they can’t ever admit that PyPy’s approach had any merits. It is no accident that PyPy’s team produced like three working JIT frameworks while none of the CPython forks could match them: CPython is architecturally slow.
Source: I still remember chatting with GvR about this like a decade ago at a Chinese restaurant midway through a PyCon.
From https://julialang.org/ [julialang.org], first paragraph:
> Julia was designed from the beginning for high performance. Julia programs compile to efficient native code for multiple platforms via LLVM.
Turns out, if you set out to make something fast, and you use the excellent optimization and code generation abilities of LLVM, it is easy to come up with something faster than something that wasn’t designed or built to be fast.
“Interpreted” isn’t really a feature of a language anymore, now that most “interpreted languages” are actually JIT compiled. Look at Javascript, which started out interpreted but has had a ton of work put into making it faster. Features of the language do affect performance, or at least make it much easier to optimize, such as being statically typed. And if you want to write super optimized code, it’s essential to have features like precise control of memory layout and explicit vectorization. But you ca
There is still quite a difference in run times between interpreted and compiled languages https://benchmarksgame-team.pa… [debian.net]
Pypy is garbage collected so one must write code to be Pypy compatible. I agree it is not a big deal though.
Some slowdown also results from runtime type checking (when compared to statically typed languages) and from structural typing. A dynamically typed language will still need to do some type checking in runtime (not all of it can be eliminated by JIT).
And yet there are many who find braces exceedingly ugly. I’m never quite sure exactly where to place them. Do I go for what looks best and is most readable, or do I try to make the matching braces line up on the same column?
The most popular style, K&R or derivative, has you place the opening brace on the same line as the statement which looks the best to me, but then you’ve lost the ability to quickly visually line up opening and closing braces with your eyes. So at that point a casual glance at the c
> so why not just ditch the braces and make it formal.
1. Indentation is NOT enough to tell intent. Given:
if foo:
x = y+foo
if x > 4:
foo =3

Was the original block:
if foo:
x = y+foo
if x > 4:
foo =3

O?
if foo:
x = y+foo
if x > 4:
foo =3

With braces the intent is unambiguous.
2. Indentation does NOT change functionality. Image if a Mathematician said “a = b + c” means something different then “a=b+c”. They would be laughed
I don’t understand your point. You show two versions of the code that are indented differently. They do different things, and in each case the indentation makes it totally clear what it does. So what’s your point?

2. Indentation does NOT change functionality. Image if a Mathematician said “a = b + c” means something different then “a=b+c”. They would be laughed out of the room for being an idiot.

2. Indentation does NOT change functionality. Image if a Mathematician said “a = b + c” means something different then “a=b+c”. They would be laughed out of the room for being an idiot.
You first said “indentation”, then gave an example that doesn’t involve indentation. So again, what’s your point? Are you saying no whitespace character should ever affect meaning? Tryremovingallthewhitespacecharactersfromsometextandseeifitmatters.
> and in each case the indentation makes it totally clear what it does. So what’s your point?
Good luck emailing / copy-pasting when whitespace is trashed and those latter two both turn into the first case.
With the always-use-braces style [c2.com] I can copy-paste and not worry IF the whitespace gets mangled / dropped / condensed down to just 1 consecutive space like in HTML.
> Are you saying no whitespace character should ever affect meaning?
You’re assuming NO whitespace equivalent to SOME whitespace. While th
There may be more comments in this discussion. Without JavaScript enabled, you might want to turn on Classic Discussion System in your preferences instead.
Behavior of Star Clusters Challenges Newton’s Laws of Gravity
Apple’s $100 Million ‘Small Developer Assistance Fund’ Surprises Developers With Payouts
There are bugs and then there are bugs. And then there are bugs. — Karl Lehenbauer

source

Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles