I hope there could be an official tool used to transfer python code to rust code automatically?

[dbsxdbsx]

(I am not pretty sure if the below suggestion could be treated as a RFC, but I think it is useful for rust development.)

We know Rust aims to be a stable and productive programming language. But at present, its ecosystem lacks a lot of wheels. Meanwhile, Python is on the very opposite position. Therefore, I think it would make Rust more productive if Rust could absorb Python's ecosystem(many mature wheels) as soon as possible.

I know there are many people trying to make wheels to enrich Rust ecosystem, some for python users. But there are still some problems for people to transfer Python code to Rust. Let's take the popular Python module numpy as an example:

1. indeed, there are some crates try to simulate numpy module. BUT, people would not be confident enough to use them as they are not official supported by numpy team.
2. The time: it would take quite a long time to make them mature enough as they are in Python, even the corresponding Rust crates are supported officially by original python module team?
3. No ganrantee on support for teams those built many popular Python modules to make corresponding crates in Rust.

So, I am asking if it is possible to build a tool like Rustfmt OFFICIALLY by Rust team, with witch could be used to automatically tranpile any Python script code（including module used in script） into corresponding Rust code, like pyrs does?

With such a tool:

1. it is possible to program in Python for checking logic first, then transpile it automatically to Rust to make the whole programming routine efficient enough--- this routine, at least I think is the best way to do programming constructively and productively.
2. people would be confident enough to use it as it is officialy supported.
3. This saves people a lot of time, instead of spending time on building crates that will be unstable for quite a long time.

Someone would ask why I stick to only Python?

1. Python is the most popular used language with the best ecosystem, the ecosystem feature is exactly Rust lacks MOST.
2. For those compiled languge, like C++， there is always a way to use wheels from their field through FFI, let alone some crates to do it , like cxx. But Python is an interpreted language with an official interpreter contains GIL，even with some crates, like pyo3, it still makes Rust program not efficient enough due to GIL.

[Lokathor]

This is effectively impossible to do automatically.

However, if you think that it can be done, start doing it yourself. Any sufficiently complete open source tool can be adopted by the rust project later on if it works out (as long as it uses the same license as the rest of the rust project: Apache2/MIT)

[SOF3]

Nothing stops you from creating an FFI interface or an RMI service such that a Rust program can communicate with a Python runtime. You don't have to RewriteItInRust or TranspileItToRust to use Python libraries in Rust.

[dbsxdbsx]

@Lokathor, what do you mean by "effectively" impossible? Actually, I post this suggestion here, in that I think this tool (if it is not technically impossible) would be more meaningful to be maintained by an official team--- so that more people would know it and then contribute it, and it would be easier to communicate with python team if there are technique issues need to solve for bridging these 2 languages.

@SOF3，As what I've stated, the Python runtime has a GIL, so it is not a good option in multi-thread programming in rust with python module, and I don't think it a good idea to deploy a rust-based programme with an python interpreter. Othewise, I would use Pyo3.

[Lokathor]

I mean that the difficulty of the converter project would be so high that it would be actually easier to just write each Python program from scratch in Rust, and you'll have useful results faster that way.

[dbsxdbsx]

@Lokathor, Indeed, I've already felt difficult to do the auto-transpiling job before posting this "issue", when I realize the difference between these 2 languages, like the class, attributes, decoration concepts those only exist in python. But I still think doing so (auto-tanspiling) is more cost-effective than writing every python module in rust from scratch---I mean it does not cost-effective if only seldom python modules needs to be rewritten, but it does cost effective when lots of python modules needs to be rewritten.

By the way, I am also aspired by work from @hegza, relating to this issue.

Anyway, if most of members in rust team think this suggestion is too naive and unpractical, please close this issue.

[burdges]

NumPy is 35% C. SciPy is 48% C, C++, and Fortran. All this code either links to or replicates older C, C++, and Fortran system libraries which come with typical Linux distributions. If you need those C, C++, or Fortran dependencies then use them directly from Rust.

Any transpiler output requires manual reworking before being useful and a complete rewrite before being idiomatic enough for a public interface. There do exist languages specifically designed around a transpiler-like phase, like doge or scala, but even they only transpile behind the scenes and never produce idiomatic code.

If one first accepts that transpilation means a computer assisted human rewrite, then one could develop independently useful interactive refactoring aids for VSCode that simplify some tasks, like choosing among borrows ala T, Box<T>, Gc<T>, RefCell<T>, Arc<Mutex<T>>, etc. or [T], [T; N], Vec<T>, ArrayVec<T,N>, Arc<[T]>, GcVec<T>, etc., or maybe do deeply cross linked unit tests and fuzzing.

[michaelb]

@dbsxdbsx The tool you mentionned is useful and all, but it's more like a 'google translate' doing word-by-word work. It will not produce as good a translation as a human, and that just won't cut it especially for Rust. That's why a developer has to go fix (and check!) the translated code in the linked project.

Some (most) python programs will never be 'automagically' translated to Rust because their core design relies on things that can't work in Rust, for safety or ownership reasons, or just that the concept (multi-inheritance for example) has no equivalent.

It's not that you can't write some types of algorithms in Rust. You can. But it sometimes requires completely different design choices, which an 'automagic' py2rust program will never be able to guess.

[SOF3]

@dbsxdbsx Python code translated to Rust would almost never be efficient, especially when high-level concepts like inheritance are involved. The performance of Rust largely comes from code written idiomatically for it, e.g. using enums instead of dynamic dispatch. And as @michaelb mentioned, there are some intrinsically incompatible things that cannot be transpiled. For example, most OOP designs (not only Python, but also Java etc.) involve cyclic dependencies, and this is a pattern that is almost impossible in Rust. Furthermore, nothing stops you from connecting to multiple python runtimes if you are concerned about multithreading.

[scottmcm]

I'll note that it's substantially easier to do C→Rust, but the projects for that aren't official either:

• https://github.com/immunant/c2rust
• https://github.com/jameysharp/corrode

I agree with @Lokathor here -- there's nothing about this project I can see that requires that it be official. That's especially true for Rust, given that core things used pervasively (like regex and serde) aren't "OFFICIALLY by Rust team" either.

[DoumanAsh]

There is no need for this shit

[jhpratt]

@DoumanAsh Please be respectful.

[adsharma]

https://github.com/adsharma/py2many is based on pyrs, comes with more included tests, improved type inference and more modular (AST rewriters, API plugins etc).

It's true that you can't transpile every python program and it's a non goal. We're trying to document the supported subset here:

https://github.com/adsharma/py2many/blob/main/doc/langspec.md

About to make a 0.3 release in the next few days. Available from pypi.

[mardab]

You have my attention.

As you can see, fully automatic conversion is near-impossible, but there are things you can do.
Take a look at RustPython and PyO3, the 2 approaches to running Python under Rust and see what you can do yourself, since there is no RFC that stops you from writing a transpiler crate.

With GIL, however, there's a problem, since it is inherent to CPython and all those mature packages written for it. PyPy avoids this problem with it's own fork of numpy, that (among other things) does not call C functions underneath.

So, to put it bluntly, if you want numpy in Rust, call its libraries directly throgh FFI, though it is unsafe, so might as well work on RIIR, I'm sure there's crate for that in the works already.

[dbsxdbsx]

@mardab, thanks for your reply.Actually, I've known PyO3 and RustPython before posting this topic. And if RustPython or PyPy would be treated as official python intepreter, there would be no GIL issue.

[adsharma]

I want to re-examine the thesis of this RFC

it would make Rust more productive if Rust could absorb Python's ecosystem(many mature wheels) as soon as possible.

Claim: 100% python compatibility with support for code written to GIL isn't going to make rust more productive.

Instead, supporting a large enough subset of python to write real world programs and reuse much of the python ecosystem libraries is what's going to be interesting.

This is what makes py2many different from pypy, cython, nuitka and mypyc. Would love to hear more about the language spec:

https://github.com/adsharma/py2many/blob/main/doc/langspec.md

In order for this vision to work, python ecosystem also needs to change:

• Some libraries written in Python C-API need to be rewritten in the above dialect of python and transpiled. So more pure python in stdlib.

• Python will need more functional programming APIs to cover all the critical features of rust. Pattern matching as an expression is a big one (Pattern Matching as an expression py2many/py2many#407)

py2many.py --rust=1 --extension test.py

has experimental support for generating a PyO3 extension.

Please chime in here if you have more feedback on the language spec: py2many/py2many#205