I’d like to call my cdef methods and improve the speed of my program as much as possible. I do not want to use cpdef (I explain why below). Ultimately, I’d like to access cdef methods (some of which return void) that are members of my Cython extensions.
I tried following this example, which gives me the impression that I can call a cdef function by making a Python (def) wrapper for it.
I can’t reproduce these results, so I tried a different problem for myself (summing all the numbers from 0 to n).
Of course, I’m looking at the documentation, which says
The directive cpdef makes two versions of the method available; one fast for use from Cython and one slower for use from Python.
and later (emphasis mine),
This does slightly more than providing a python wrapper for a cdef method: unlike a cdef method, a cpdef method is fully overridable by methods and instance attributes in Python subclasses. It adds a little calling overhead compared to a cdef method.
So how does one use a cdef function without the extra calling overhead of a cpdef function?
With the code at the end of this question, I get the following results:
def/cdef: 273.04207632583245 def/cpdef: 304.4114626176919 cpdef/cdef: 0.8969507060538783
Somehow, cpdef is faster than cdef. For n < 100, I can occasionally get cpdef/cdef > 1, but it’s rare. I think it has to do with wrapping the cdef function in a def function. This is what the example I link to does, but they claim better performance from using cdef than from using cpdef.
I’m pretty sure this is not how you wrap a cdef function while avoiding the additional overhead (the source of which is not clearly documented) of a cpdef.
And now, the code:
setup.py
from setuptools import setup, Extension from Cython.Build import cythonize pkg_name = "tmp" compile_args=['-std=c++17'] cy_foo = Extension( name=pkg_name + '.core.cy_foo', sources=[ pkg_name + '/core/cy_foo.pyx', ], language='c++', extra_compile_args=compile_args, ) setup( name=pkg_name, ext_modules=cythonize(cy_foo, annotate=True, build_dir='build'), packages=[ pkg_name, pkg_name + '.core', ], )
foo.py
def foo_def(n): sum = 0 for i in range(n): sum += i return sum
cy_foo.pyx
def foo_cdef(n): return foo_cy(n) cdef int foo_cy(int n): cdef int sum = 0 cdef int i = 0 for i in range(n): sum += i return sum cpdef int foo_cpdef(int n): cdef int sum = 0 cdef int i = 0 for i in range(n): sum += i return sum
test.py
import timeit from tmp.core.foo import foo_def from tmp.core.cy_foo import foo_cdef from tmp.core.cy_foo import foo_cpdef n = 10000 # Python call start_time = timeit.default_timer() a = foo_def(n) pyTime = timeit.default_timer() - start_time # Call Python wrapper for C function start_time = timeit.default_timer() b = foo_cdef(n) cTime = timeit.default_timer() - start_time # Call cpdef function, which does more than wrap a cdef function (whatever that means) start_time = timeit.default_timer() c = foo_cpdef(n) cpTime = timeit.default_timer() - start_time print("def/cdef:") print(pyTime/cTime) print("def/cpdef:") print(pyTime/cpTime) print("cpdef/cdef:") print(cpTime/cTime)
