いろいろ備忘録日記

主に .NET とか Go とか Flutter とか Python絡みのメモを公開しています。

Codon (high-performance Python compiler, MITが発表)

python

概要

以下、自分用のメモです。忘れないうちにメモメモ。。

CodonというPythonコンパイラがあるみたいですね。知らなかったです。

C,C++に匹敵する最適化が出来る可能性があるとのこと。

面白そうですね。

CodonのGithubリポジトリのREADMEには以下のように説明されています。

Codon is a high-performance Python compiler that compiles Python code to native machine code without any runtime overhead.

また、以下の記載もあります。

Unlike Python, Codon supports native multithreading, which can lead to speedups many times higher still.

以下、参照リソースをメモメモ。。

github.com

docs.exaloop.io

zenn.dev

engineer.fabcross.jp

dev.classmethod.jp

www.itmedia.co.jp

ライセンスについて

https://docs.exaloop.io/codon/general/faq#is-codon-free

に、以下のQAがあります。

Is Codon free?

Codon is and always will be free for non-production use. That means you can use Codon freely for personal, academic, or other non-commercial applications.

「Codonは、非商業利用において常に無料です。」ってことみたいですね。

とりあえずインストールを試してみる

Gitpod上で試してみました。

gitpod /workspace/playground (master) $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:        22.04
Codename:       jammy

とりあえず、github リポジトリに書いてある通りにやってみる。

gitpod /workspace/playground (master) $ /bin/bash -c "$(curl -fsSL https://exaloop.io/install.sh)"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
codon-deploy/python/
codon-deploy/python/pyproject.toml
codon-deploy/python/codon_jit.egg-info/
codon-deploy/python/codon_jit.egg-info/requires.txt
codon-deploy/python/codon_jit.egg-info/PKG-INFO
codon-deploy/python/codon_jit.egg-info/dependency_links.txt
codon-deploy/python/codon_jit.egg-info/top_level.txt
codon-deploy/python/codon_jit.egg-info/SOURCES.txt
codon-deploy/python/setup.py
codon-deploy/python/README.md
codon-deploy/python/codon/
codon-deploy/python/codon/jit.pyx
codon-deploy/python/codon/jit.pxd
codon-deploy/python/codon/__init__.py
codon-deploy/python/codon/decorator.py
codon-deploy/python/codon/version.py
codon-deploy/python/MANIFEST.in
codon-deploy/python/.gitignore
codon-deploy/bin/
codon-deploy/bin/codon
codon-deploy/include/
codon-deploy/include/fmt/
codon-deploy/include/fmt/compile.h
codon-deploy/include/fmt/ostream.h
codon-deploy/include/fmt/os.h
codon-deploy/include/fmt/printf.h
codon-deploy/include/fmt/chrono.h
codon-deploy/include/fmt/xchar.h
codon-deploy/include/fmt/color.h
codon-deploy/include/fmt/std.h
codon-deploy/include/fmt/ranges.h
codon-deploy/include/fmt/args.h
codon-deploy/include/fmt/core.h
codon-deploy/include/fmt/format.h
codon-deploy/include/fmt/format-inl.h
codon-deploy/include/codon/
codon-deploy/include/codon/compiler/
codon-deploy/include/codon/compiler/compiler.h
codon-deploy/include/codon/compiler/jit_extern.h
codon-deploy/include/codon/compiler/engine.h
codon-deploy/include/codon/compiler/error.h
codon-deploy/include/codon/compiler/debug_listener.h
codon-deploy/include/codon/compiler/memory_manager.h
codon-deploy/include/codon/compiler/jit.h
codon-deploy/include/codon/cir/
codon-deploy/include/codon/cir/transform/
codon-deploy/include/codon/cir/transform/parallel/
codon-deploy/include/codon/cir/transform/parallel/schedule.h
codon-deploy/include/codon/cir/transform/parallel/openmp.h
codon-deploy/include/codon/cir/transform/pythonic/
codon-deploy/include/codon/cir/transform/pythonic/dict.h
codon-deploy/include/codon/cir/transform/pythonic/generator.h
codon-deploy/include/codon/cir/transform/pythonic/io.h
codon-deploy/include/codon/cir/transform/pythonic/str.h
codon-deploy/include/codon/cir/transform/pythonic/list.h
codon-deploy/include/codon/cir/transform/rewrite.h
codon-deploy/include/codon/cir/transform/pass.h
codon-deploy/include/codon/cir/transform/manager.h
codon-deploy/include/codon/cir/transform/folding/
codon-deploy/include/codon/cir/transform/folding/folding.h
codon-deploy/include/codon/cir/transform/folding/const_prop.h
codon-deploy/include/codon/cir/transform/folding/rule.h
codon-deploy/include/codon/cir/transform/folding/const_fold.h
codon-deploy/include/codon/cir/transform/lowering/
codon-deploy/include/codon/cir/transform/lowering/imperative.h
codon-deploy/include/codon/cir/transform/lowering/pipeline.h
codon-deploy/include/codon/cir/transform/cleanup/
codon-deploy/include/codon/cir/transform/cleanup/global_demote.h
codon-deploy/include/codon/cir/transform/cleanup/dead_code.h
codon-deploy/include/codon/cir/transform/cleanup/replacer.h
codon-deploy/include/codon/cir/transform/cleanup/canonical.h
codon-deploy/include/codon/cir/cir.h
codon-deploy/include/codon/cir/llvm/
codon-deploy/include/codon/cir/llvm/optimize.h
codon-deploy/include/codon/cir/llvm/llvisitor.h
codon-deploy/include/codon/cir/llvm/gpu.h
codon-deploy/include/codon/cir/llvm/llvm.h
codon-deploy/include/codon/cir/base.h
codon-deploy/include/codon/cir/flow.h
codon-deploy/include/codon/cir/module.h
codon-deploy/include/codon/cir/const.h
codon-deploy/include/codon/cir/dsl/
codon-deploy/include/codon/cir/dsl/codegen.h
codon-deploy/include/codon/cir/dsl/nodes.h
codon-deploy/include/codon/cir/types/
codon-deploy/include/codon/cir/types/types.h
codon-deploy/include/codon/cir/analyze/
codon-deploy/include/codon/cir/analyze/module/
codon-deploy/include/codon/cir/analyze/module/side_effect.h
codon-deploy/include/codon/cir/analyze/module/global_vars.h
codon-deploy/include/codon/cir/analyze/analysis.h
codon-deploy/include/codon/cir/analyze/dataflow/
codon-deploy/include/codon/cir/analyze/dataflow/reaching.h
codon-deploy/include/codon/cir/analyze/dataflow/cfg.h
codon-deploy/include/codon/cir/analyze/dataflow/dominator.h
codon-deploy/include/codon/cir/analyze/dataflow/capture.h
codon-deploy/include/codon/cir/util/
codon-deploy/include/codon/cir/util/operator.h
codon-deploy/include/codon/cir/util/packs.h
codon-deploy/include/codon/cir/util/side_effect.h
codon-deploy/include/codon/cir/util/context.h
codon-deploy/include/codon/cir/util/outlining.h
codon-deploy/include/codon/cir/util/irtools.h
codon-deploy/include/codon/cir/util/matching.h
codon-deploy/include/codon/cir/util/cloning.h
codon-deploy/include/codon/cir/util/inlining.h
codon-deploy/include/codon/cir/util/iterators.h
codon-deploy/include/codon/cir/util/format.h
codon-deploy/include/codon/cir/util/visitor.h
codon-deploy/include/codon/cir/value.h
codon-deploy/include/codon/cir/func.h
codon-deploy/include/codon/cir/pyextension.h
codon-deploy/include/codon/cir/var.h
codon-deploy/include/codon/cir/attribute.h
codon-deploy/include/codon/cir/instr.h
codon-deploy/include/codon/parser/
codon-deploy/include/codon/parser/ast/
codon-deploy/include/codon/parser/ast/error.h
codon-deploy/include/codon/parser/ast/types.h
codon-deploy/include/codon/parser/ast/types/
codon-deploy/include/codon/parser/ast/types/union.h
codon-deploy/include/codon/parser/ast/types/static.h
codon-deploy/include/codon/parser/ast/types/traits.h
codon-deploy/include/codon/parser/ast/types/class.h
codon-deploy/include/codon/parser/ast/types/type.h
codon-deploy/include/codon/parser/ast/types/function.h
codon-deploy/include/codon/parser/ast/types/link.h
codon-deploy/include/codon/parser/ast/expr.h
codon-deploy/include/codon/parser/ast/stmt.h
codon-deploy/include/codon/parser/ast.h
codon-deploy/include/codon/parser/ctx.h
codon-deploy/include/codon/parser/peg/
codon-deploy/include/codon/parser/peg/rules.h
codon-deploy/include/codon/parser/peg/peg.h
codon-deploy/include/codon/parser/common.h
codon-deploy/include/codon/parser/visitors/
codon-deploy/include/codon/parser/visitors/format/
codon-deploy/include/codon/parser/visitors/format/format.h
codon-deploy/include/codon/parser/visitors/translate/
codon-deploy/include/codon/parser/visitors/translate/translate.h
codon-deploy/include/codon/parser/visitors/translate/translate_ctx.h
codon-deploy/include/codon/parser/visitors/simplify/
codon-deploy/include/codon/parser/visitors/simplify/ctx.h
codon-deploy/include/codon/parser/visitors/simplify/simplify.h
codon-deploy/include/codon/parser/visitors/doc/
codon-deploy/include/codon/parser/visitors/doc/doc.h
codon-deploy/include/codon/parser/visitors/typecheck/
codon-deploy/include/codon/parser/visitors/typecheck/ctx.h
codon-deploy/include/codon/parser/visitors/typecheck/typecheck.h
codon-deploy/include/codon/parser/visitors/visitor.h
codon-deploy/include/codon/parser/cache.h
codon-deploy/include/codon/app/
codon-deploy/include/codon/config/
codon-deploy/include/codon/config/config.h
codon-deploy/include/codon/dsl/
codon-deploy/include/codon/dsl/dsl.h
codon-deploy/include/codon/dsl/plugins.h
codon-deploy/include/codon/runtime/
codon-deploy/include/codon/runtime/lib.h
codon-deploy/include/codon/util/
codon-deploy/include/codon/util/jupyter.h
codon-deploy/include/codon/util/common.h
codon-deploy/include/peglib.h
codon-deploy/lib/
codon-deploy/lib/codon/
codon-deploy/lib/codon/libcodonrt.so
codon-deploy/lib/codon/stdlib/
codon-deploy/lib/codon/stdlib/unittest.codon
codon-deploy/lib/codon/stdlib/time.codon
codon-deploy/lib/codon/stdlib/sortedlist.codon
codon-deploy/lib/codon/stdlib/datetime.codon
codon-deploy/lib/codon/stdlib/algorithms/
codon-deploy/lib/codon/stdlib/algorithms/timsort.codon
codon-deploy/lib/codon/stdlib/algorithms/qsort.codon
codon-deploy/lib/codon/stdlib/algorithms/insertionsort.codon
codon-deploy/lib/codon/stdlib/algorithms/heapsort.codon
codon-deploy/lib/codon/stdlib/algorithms/pdqsort.codon
codon-deploy/lib/codon/stdlib/algorithms/strings.codon
codon-deploy/lib/codon/stdlib/typing.codon
codon-deploy/lib/codon/stdlib/operator.codon
codon-deploy/lib/codon/stdlib/python.codon
codon-deploy/lib/codon/stdlib/cmath.codon
codon-deploy/lib/codon/stdlib/threading.codon
codon-deploy/lib/codon/stdlib/bisect.codon
codon-deploy/lib/codon/stdlib/random.codon
codon-deploy/lib/codon/stdlib/gzip.codon
codon-deploy/lib/codon/stdlib/os/
codon-deploy/lib/codon/stdlib/os/__init__.codon
codon-deploy/lib/codon/stdlib/os/path.codon
codon-deploy/lib/codon/stdlib/experimental/
codon-deploy/lib/codon/stdlib/experimental/simd.codon
codon-deploy/lib/codon/stdlib/pickle.codon
codon-deploy/lib/codon/stdlib/copy.codon
codon-deploy/lib/codon/stdlib/re.codon
codon-deploy/lib/codon/stdlib/statistics.codon
codon-deploy/lib/codon/stdlib/math.codon
codon-deploy/lib/codon/stdlib/openmp.codon
codon-deploy/lib/codon/stdlib/string.codon
codon-deploy/lib/codon/stdlib/itertools.codon
codon-deploy/lib/codon/stdlib/getopt.codon
codon-deploy/lib/codon/stdlib/gpu.codon
codon-deploy/lib/codon/stdlib/functools.codon
codon-deploy/lib/codon/stdlib/sys.codon
codon-deploy/lib/codon/stdlib/heapq.codon
codon-deploy/lib/codon/stdlib/internal/
codon-deploy/lib/codon/stdlib/internal/core.codon
codon-deploy/lib/codon/stdlib/internal/python.codon
codon-deploy/lib/codon/stdlib/internal/khash.codon
codon-deploy/lib/codon/stdlib/internal/c_stubs.codon
codon-deploy/lib/codon/stdlib/internal/__init_test__.codon
codon-deploy/lib/codon/stdlib/internal/static.codon
codon-deploy/lib/codon/stdlib/internal/format.codon
codon-deploy/lib/codon/stdlib/internal/file.codon
codon-deploy/lib/codon/stdlib/internal/__init__.codon
codon-deploy/lib/codon/stdlib/internal/types/
codon-deploy/lib/codon/stdlib/internal/types/complex.codon
codon-deploy/lib/codon/stdlib/internal/types/slice.codon
codon-deploy/lib/codon/stdlib/internal/types/error.codon
codon-deploy/lib/codon/stdlib/internal/types/collections/
codon-deploy/lib/codon/stdlib/internal/types/collections/set.codon
codon-deploy/lib/codon/stdlib/internal/types/collections/dict.codon
codon-deploy/lib/codon/stdlib/internal/types/collections/list.codon
codon-deploy/lib/codon/stdlib/internal/types/collections/tuple.codon
codon-deploy/lib/codon/stdlib/internal/types/generator.codon
codon-deploy/lib/codon/stdlib/internal/types/bool.codon
codon-deploy/lib/codon/stdlib/internal/types/array.codon
codon-deploy/lib/codon/stdlib/internal/types/float.codon
codon-deploy/lib/codon/stdlib/internal/types/int.codon
codon-deploy/lib/codon/stdlib/internal/types/ptr.codon
codon-deploy/lib/codon/stdlib/internal/types/range.codon
codon-deploy/lib/codon/stdlib/internal/types/strbuf.codon
codon-deploy/lib/codon/stdlib/internal/types/byte.codon
codon-deploy/lib/codon/stdlib/internal/types/optional.codon
codon-deploy/lib/codon/stdlib/internal/types/intn.codon
codon-deploy/lib/codon/stdlib/internal/types/str.codon
codon-deploy/lib/codon/stdlib/internal/internal.codon
codon-deploy/lib/codon/stdlib/internal/sort.codon
codon-deploy/lib/codon/stdlib/internal/attributes.codon
codon-deploy/lib/codon/stdlib/internal/builtin.codon
codon-deploy/lib/codon/stdlib/internal/pynumerics.codon
codon-deploy/lib/codon/stdlib/internal/gc.codon
codon-deploy/lib/codon/stdlib/internal/dlopen.codon
codon-deploy/lib/codon/stdlib/internal/str.codon
codon-deploy/lib/codon/stdlib/collections.codon
codon-deploy/lib/codon/libomp.so
codon-deploy/lib/codon/libcodon_jupyter.so
codon-deploy/lib/codon/plugins/
codon-deploy/lib/codon/libcodonc.so
100 43.1M  100 43.1M    0     0  23.6M      0  0:00:01  0:00:01 --:--:-- 38.3M
codon-deploy/lib64/
codon-deploy/lib64/libfmt.a
codon-deploy/lib64/cmake/
codon-deploy/lib64/cmake/fmt/
codon-deploy/lib64/cmake/fmt/fmt-config.cmake
codon-deploy/lib64/cmake/fmt/fmt-targets-release.cmake
codon-deploy/lib64/cmake/fmt/fmt-targets.cmake
codon-deploy/lib64/cmake/fmt/fmt-config-version.cmake
codon-deploy/lib64/pkgconfig/
codon-deploy/lib64/pkgconfig/fmt.pc
PATH export command:
  export PATH=/home/gitpod/.codon/bin:$PATH
Update PATH in /home/gitpod/.bash_profile? [y/n] y
Updating /home/gitpod/.bash_profile
Codon successfully installed at: /home/gitpod/.codon
Open a new terminal session or update your PATH to use codon

Codon successfully installed at: /home/gitpod/.codon

Open a new terminal session or update your PATH to use codon

って出てるのでインストールは成功したみたいですね。

新しいターミナル開いて、パスに存在するかを確認。

gitpod /workspace/playground (master) $ which codon
/home/gitpod/.codon/bin/codon

オッケイ。

とりあえずへろーワールド

Codonにすることによって、効果がでるようなプログラムについては上の参照リンクの記事様の方で記載されていますので、そちら参照する方が良いですね。

私は凡人なので、何事もまずへろーワールドです。

#!/usr/bin/env python3
# coding: utf-8

import threading as thr

for i in range(10):
    print(f'hello world {thr.get_ident()}')

まずは、普通に python で実行しましょう。

gitpod /workspace/playground (master) $ python -V
Python 3.10.7

gitpod /workspace/playground (master) $ python helloworld.py 
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872
hello world 140658955351872

CPythonは、GILがあるので実行すると当然上のようになります。

Codonは、GILの縛りはないので、違う結果になるはず。

gitpod /workspace/playground (master) $ codon -version
0.16.0

gitpod /workspace/playground (master) $ codon run helloworld.py 
hello world 1
hello world 1
hello world 1
hello world 1
hello world 1
hello world 1
hello world 1
hello world 1
hello world 1
hello world 1

ありゃ。ならないですね。てことで、ドキュメントを見るとOpenMP経由で並列機能を利用したい場合は @par デコレータをつけろと書いてありました。

元のサンプルを以下のようにします。

#!/usr/bin/env python3
# coding: utf-8

import threading as thr

@par(num_threads=8)
for i in range(10):
    print(f'hello world {thr.get_ident()}')

再び実行。

gitpod /workspace/playground (master) $ codon run helloworld.py 
hello world 2
hello world 2
hello world 5
hello world 8
hello world 1
hello world 1
hello world 6
hello world 3
hello world 4
hello world 7

ちゃんとスレッド出来てますね。

あとは、ビルドしてみましょう。

gitpod /workspace/playground (master) $ ls -l helloworld*
-rw-r--r-- 1 gitpod gitpod 149 May 21 12:54 helloworld.py


gitpod /workspace/playground (master) $ codon build helloworld.py 
gitpod /workspace/playground (master) $ ls -l helloworld*
-rwxr-xr-x 1 gitpod gitpod 122104 May 21 13:00 helloworld
-rw-r--r-- 1 gitpod gitpod    149 May 21 12:54 helloworld.py

gitpod /workspace/playground (master) $ file ./helloworld
./helloworld: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=dde2a5907ba68cb2697796d9703f523089d0da57, for GNU/Linux 3.2.0, with debug_info, not stripped


gitpod /workspace/playground (master) $ ./helloworld 
hello world 8
hello world 7
hello world 3hello world 6
hello world 4

hello world 2
hello world 2
hello world 5
hello world 1
hello world 1

ちゃんとネイティブコンパイル出来ていますね。

参考情報

過去の記事については、以下のページからご参照下さい。

サンプルコードは、以下の場所で公開しています。