Hacker News
points by rich_sasha 2 days ago | 0 comments
Here’s an idea instead (maybe it already exists). Allow a decorator for writing inline Cython. It solves, I think, 90% of the problem. On first run, or eagerly, compile the decorated functions. Basically, Cython is the JIT, and user enables it explicitly.

This way I can easily write inlined functions with proper C speed, in a Python esque language.

Reasons why I rarely use Cython without this:

- unmodified Python code running in Cython is often faster, but not that much. 50% or so is my rule of thumb.

- but the development style is very different for Cython and Python, adding a lot of impedance. Eg reloading doesn’t quite work, profiling is different, you need a build directory and a build step at deployment etc.

- Cython proper is different enough and I’d argue unintuitive enough that if I have to have all these things in place, I might as well wrap my function in Rust.

Such a decorator would give you a low friction way to make things faster when you need it, with scope for making it much faster via all the funny Cython, while keeping Python-esque semantics, in particular "development semantics".

I think it would require some language modification since Cython was definitely a superset of Python; I think some of that syntax has been streamlined, but maybe still a strict superset.

[–] somethingsome 2 days ago
I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.

On another note, I'm working on a (big) PEP for python, and looking for maintainers and sponsors (as per python's rules). Do someone know where I could find such amazing persons? (I already have the code, works in cython,graalVM and pypy, and it's kind of a big change but I believe it will help a lot the language)

[–] ModernMech 2 days ago
However because of this, Julia has has persistent problems with "time to first plot", which has gotten better, but they cannot fully ameliorate due to the way it fundamentally works. So there are tradeoffs -- the performance gets better, but the interpreted REPL experience is markedly worse.
[–] FacelessJim 2 days ago
By the way, the current nightly (especially if considering the optimized build) has a very substantial TTFX drop compared to the current release [1]

[1] https://perf.julialang.org/?tab=overview

[–] StefanKarpinski 22 hours ago
It's not unavoidable, it just hasn't yet been avoided. It could be avoided the same way as other JITs: by interpreting until a compiled version of the code is available. That's actually being worked on, but it adds considerable complexity. Among other things, it requires being able to jump out of an executing interpreter loop into an optimized, compiled version of the same function. Doable, but annoying to implement and requires compiling two versions of the function instead of one. It can also cause performance instability — see https://tratt.net/laurie/blog/2022/more_evidence_for_problem... and the older paper referenced there.
[–] pjmlp 2 days ago
It is overly exaggerated, especially when taken into account how long it takes to compile native libraries that people keep calling "Python" libraries, if there isn't a suitable wheel for the current environment.
[–] ModernMech 2 days ago
I think it's true that it's exaggerated, but that doesn't mean the tradeoff doesn't exist.
[–] pjmlp 2 days ago
A tradeoff that doesn't make sense to complain about, when the alternative is waiting for native libraries being compiled, in a mix of C, C++, Fortran and more recently Rust.

A pain point that is big enough that at least two programming languages, Julia and Mojo, were created to move beyond it.

Which would not even be an issue, if Lisp, Scheme and Prolog had not lost their aura among AI devs.

[–] ModernMech 2 days ago
Julia and Mojo are attempts at solving the "two language problem", which not every Python user actually has -- they're fine doing work exclusively in Python, not even using any of the libraries you're talking about.

Mojo in particular recognized to compete with Python they'd have to be a superset, then they backed of that goal when they realized it would be too hard.

So at the end of the day, Python has a value proposition and it cannot be entirely subsumed by other languages, because those languages make tradeoffs that move away from Python's value.

[–] pjmlp 2 days ago
Python's value is being a portable scripting language, a better Perl, unfortunately some try to do full blown applications with an interpreted language.
[–] zahlman 2 days ago
> I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.

Python already compiles to bytecode (the .pyc files are a cache thereof). It's just that interpreting even the simple bytecode instructions can take dozens of clock cycles each. I imagine that a lot of this is not in the interpreter loop (which has seen quite a bit of optimization, IIRC up to the level of "predictive" gotos within a big switch statement within a loop) but just unwrapping objects and moving values to and from registers, dealing with the fact that there is basically never any cache coherency for anything etc.

Basically I'm not convinced that writing out the set of machine-code instructions needed to interpret the bytecodes would be a big win, because it takes too many instructions for simple tasks.

[–] mcepl 1 day ago
Python doesn’t really compile to *.pyc. These are just parsed tokens and AST, really not proper machine code.
[–] pjmlp 2 days ago
The Julia approach is the same as BASIC (as originally designed), Lisp, Prolog, as added info.
[–] klaff 2 days ago
I don't recall being able to inspect compiled machine code in basic like I can in Julia.

[EDIT - I assume you are commenting on Dartmouth BASIC.]

[–] pjmlp 2 days ago
And? That doesn't change the fact it was compiled into machine code before execution, nor that with exception of 8 bit home computers, there were plenty of compiled BASICs in mainframes and later 16 bit home computers.

Since we have to prove everything on Internet forums,

https://archive.org/details/bitsavers_dartmouthB_3679804/mod...

=> Saving of Compiled Programs

[–] orf 2 days ago
The language forum is the place to start- you should already have a thread there gathering feedback before writing a pep?
[–] somethingsome 2 days ago
Thank-you, it's a longtime idea, but I was finally able to implement it, but now I'm trying to learn the procedure
[–] mirashii 2 days ago
There’s an #internals-and-peps channel in the Python discord where some folks hang out.
[–] cpburns2009 2 days ago
That's basically how Python already compiles source to byte-code.
[–] jey 2 days ago
Julia gets a lot more benefit out of it because it's compiling type-specialized version of the function all the way down to machine code. So for example, a function that's being called with a `Vector{Int64}` ends up dealing with an array of unboxed ints, not going through dynamic dispatch on a generic array for every operation.
[–] cpburns2009 2 days ago
I have no doubt Julia's far better with this. I just wanted to point out Python does have a similar precedence for ahead-of-time compilation, albeit with byte-code only. If the Python steering council ever gets its act together, maybe Python could have real AOT and JIT.
[–] O5vYtytb 2 days ago
This is fine for your own application code, but it doesn't do anything for the libraries you pull in for your project. I don't think that sprinkling Cython code in all library code is going to be a popular solution, especially for those not performance focused. The idea behind a JIT is to be a general performance solution not requiring changes in existing code.
[–] rich_sasha 2 days ago
Good point.

Maybe also then, importing module as Cython?

  import django with Cython
compiles on first run?
[–] ttul 2 days ago
I built this in 2001 (https://pyinline.sourceforge.net/). Obviously, that ancient thing doesn't work anymore, but the idea worked then and it ought to work today. That being said, I think there are significantly superior options in existence now!
[–] ponco 2 days ago
The main reason I don't see it in production code is you significantly increase your chance to write seg faults that kills the runtime / behaves unexpectedly to surrounding systems
[–] stefanos82 1 day ago
If you want to use the same code unmodified *and* your code uses type hints, then you should use mypyc [1].

There are times that it generates faster code than Cython! Of course with Cython you can tweak it and push its limits with various optimizations and other techniques, but for using your existing code as it is and gain massive performance improvements, mypyc it's a must!

Code sample:

    import time

    def fib(n: int) -> int:
        if n <= 1:
            return n
        else:
            return fib(n-2) + fib(n-1)


    startTime = time.time()
    fib(35)
    duration = time.time() - startTime
    msduration = duration * 1000.0
    uduration = msduration * 1000.0
    print(f'{duration:.6f} seconds, {msduration:.2f} ms, {uduration:.2f} μs')


    $ python3 mypyfib.py 
    1.452477 seconds, 1452.48 ms, 1452477.22 μs

    $ # the same code compiled with `mypyc mypyfib.py` produces the following:
    $ python3 -c "import mypyfib"
    0.063701 seconds, 63.70 ms, 63700.91 μs

This is huge, if you think about it!

       95.61% reduction in time
     2180.15% increase in performance
       22.80x faster

[1] https://github.com/mypyc/mypyc
[–] jech 1 day ago
This benchmark only measures function call overhead, which is important but not the whole story. What kind of figures do you get with a more realistic benchmark?
[–] stefanos82 12 hours ago
No idea why my original reply is not visible, nor marked as [dead]...such a weird behavior by HN!

Here's what I have replied to you:

To be honest with you, I am not sure how to answer your question.

What I can say is that when you identify bottlenecks within a project -- whether using Cython or mypyc -- the goal is to improve code performance by optimizing the specific parts that impact the project as a whole.

Naturally, the objective is not to replace the entire codebase (switching from Python to Cython), but rather to use the right tool for the job.

In this instance, that tool is mypyc; it requires no changes to your existing code -- it simply generates a Python module that you can then import into your project.

I hope my answer satisfies you, even just by a bit.

[–] adsharma 2 days ago
This is largely the idea behind py2many + fquery, but we want to get rid of C-API as well.

  https://github.com/py2many/py2many
  https://github.com/adsharma/fquery/blob/main/tests/django_example_model.py
[–] mpweiher 2 days ago
I think the idea is correct, but hard to retrofit, you sort of have to have it there from the start.

Objective-Smalltalk runs on the Objective-C runtime, so Objective-Smalltalk methods look like Objective-C methods accessed via objc_msgSend() to the entire rest of the system, including to other Objective-Smalltalk code.

That means it doesn't really matter to the rest of the system if any particular method is interpreted, AOT-compiled or JIT-compiled. They can't tell, except for execution speed.

And of course the execution model includes machine primitives, which the interpreter also deals with but doesn't make faster, but the native compilers do.

[–] guipsp 2 days ago
This is numba?
[–] ajross 2 days ago
Yeah, this is numba. I find it really frustrating the extent to which people don't know about it.

Folks, numba is great. To first approximation: write your python function to "look like C"[1] and work over the numpy arrays you probably already have and understand, put a "@jit" decorator on it, and you're done. It JITs and runs your code natively at native performance.

With some extra effort, there are features for SIMD vectorization and thread parallelism, and even GPU backends (that, full disclosure, I've been pretty frustrated by -- but they do work).

[1] Which is to say, don't touch Python interpreter data structures. No dicts or objects or lists or strings, etc... If you're bound by the interpreter[2] you're bound by the interpreter and nothing is going to help you.

[2] A corrolary here is that the kind of C code numba is NOT good for is "following pointers between structs", which is something that can't be expressed in either python or numpy.

[–] Dylan16807 2 days ago
> No dicts or objects or lists or strings, etc.

That's a really intense restriction. Sounds like the name fits; it's not general purpose, it's jit for number crunching and little else.

I'm not sure exactly what you mean by "bound by the interpreter" but there's nothing about touching those data structures that requires the interpreter to be involved in most cases. The problem of slow APIs can be solved by the JIT having intimate knowledge of python's internals, can't it?

[–] ajross 2 days ago
> That's a really intense restriction.

Is it? An object field access in python is a dynamic hash lookup followed by a method call as a fallback (itself involving hashed lookups up the class hierarchy). That's not an "interpreter vs. native" kind of thing, that's just the semantics of the language that have to be preserved. And that's just not going to be "fast" in the way that the user of a JIT would want. Code dominated by Python datastructures is going to be "slow" in ways that JITs can't fix, ever.

Which is why Python appears to have given up on the effort in this space. PyPy too[1] really never delivered on the desired performance gains (like 5x or so on typical code) without forcing the user into careful optimization.

Numba, to its credit, doesn't even try. It falls back to python if it must for python stuff, but is quite clear that it's providing no benefit if that's what you're doing. And the stuff it does do is obvious and clean and extremely useful.

[1] PyPy is also looking pretty moribund recently.

[–] Dylan16807 2 days ago
A hash lookup can be very fast. There's no reason for it to be slow when it doesn't need to invoke the fallback. JITs can't remove the slowest case but they can make the normal case very fast. Saying it can't be fast ever is a huge cop-out.
[–] zahlman 1 day ago
Yes, Python's hash lookups are actually pretty good. The comments in the implementation are an interesting read.

But that doesn't help much if/when you're doing two hash lookups per iteration just to find two small integers to add together.

[–] ajross 2 days ago
> Saying it can't be fast ever is a huge cop-out.

Maybe, but the world is still waiting for the counterexample. Python is simply not a runtime well-suited to optimization.

In the meantime, those of us who actually enjoy real HPC problems are using numba and rolling our eyes about all the confuffery around JITs. Maybe one will arrive that proves us wrong, but numba is fast today.

[–] Dylan16807 2 days ago
LuaJIT has the same dynamism with dictionaries and it's very fast at accessing them in JIT code.

I don't have enough experience with PyPy to say much about it.

[–] zahlman 1 day ago
Probably "bound by the data model" is a better way to put it. The interpreter might not be required to touch the data structure, but you have to do a lot of touching of the data structure to get the work done (plus, its mere existence ruins any dream of cache coherency).
[–] Dylan16807 1 day ago
For most objects, you access it directly and nothing weird happens, right?

Can you not mostly cache whether anything weird needs to happen and do the direct access with minimal overhead?

You said elsewhere "But that doesn't help much if/when you're doing two hash lookups per iteration just to find two small integers to add together." but looking up two fields isn't inherently a very big overhead. And you can cache the actual hash, and if you want to you can make sure the fields on similar objects have a consistent offset and cache that offset to make it even faster. This code using lots of objects is presumably not the core number crunching on arrays; we're doing more logic per access and we can handle a few instructions to get a value.

[–] sgarland 2 days ago
I posted it as well before scrolling down and seeing yours. Deleted mine, as your post goes into much more detail.

I don’t know how people who’ve worked with Python and been frustrated with its performance don’t know about Numba.

[–] bheztio 2 days ago
Last I tried numba, admittedly a long time ago, it was a pain due to extensive system requirements or alternatively requiring use of conda, which I dislike (for many reasons, but it's besides the point - I want to use standard python tools). Did they start to bundle their dependencies or..?
[–] ajross 2 days ago
PyPI integration is clean; "pip install numba" does what you expect, uv works fine, yada yada.

The one aesthetic gripe I have is that the LLVM included is a fork. It won't even try to work with the system LLVM and insists on installing a huge wheel with custom binaries. But on a modern system an extra LLVM is noise, so I guess I don't care that much.

[–] pitrou 1 day ago
> The one aesthetic gripe I have is that the LLVM included is a fork.

It's not technically a fork, it's a standalone build of vanilla LLVM (ok, there are a couple patches: https://github.com/numba/llvmlite/tree/main/conda-recipes).

> It won't even try to work with the system LLVM and insists on installing a huge wheel with custom binaries.

Not only does LLVM not have a stable ABI, it doesn't even have a stable API. It's difficult to write code that works reliably against a non-trivial number of LLVM major versions.

For the record, here is the LLVM compatibility table for llvmlite versions (llvmlite is the Python-C++ shim that Numba uses to talk to LLVM): https://llvmlite.readthedocs.io/en/latest/#llvm-compatibilit...

If you want a Numba linked against your system LLVM, the solution would be for your distribution to provide a system package for Numba (and llvmlite).

[–] zahlman 1 day ago
I don't understand why your comment was dead; I've vouched for it.
[–] ainch 2 days ago
if you want GPU-backed/compiled numpy code then JAX is also a good option.
[–] rich_sasha 2 days ago
Not really. Numba is great at optimizing numpy code but not necessarily other general Python code.
[–] lcnmrn 2 days ago
We just need Mojo to have 99% Python friendly syntax and we can forget about CPython.
[–] __mharrison__ 2 days ago
Numba does this. Generally as fast as Cython, but much easier to write.
[–] odo1242 2 days ago
Numba is basically this.