understanding python reference counts in order to debug c-extensions -

i writing c-extension , want test in pytest.

part of testing whether reference counts correct on objects. build small test in pure python puzzles me...

from ipython get:

in [1]: x = 153513514215  in [2]: import sys  in [3]: sys.getrefcount(x) out[3]: 2 

so fare good, 1 reference assignment , 1 caller.

however following script (stackoverflow_test.py) gives following results

import sys  def test_ref_count_int():     x = 677461248192962146784178     assert sys.getrefcount(x) == 2  def test_ref_count_str():     y = 'very long , probbably unique string'     assert sys.getrefcount(y) == 2  def normal_te_st():     x = 222677461248192962146784178     y = '!!!!very long , probbably unique string!!!!'     print ('x refcount = {}'.format(sys.getrefcount(x)))     print ('y refcount = {}'.format(sys.getrefcount(y)))  if __name__ == '__main__':     normal_te_st() 

when run normal python script

$ python3 stackoverflow_test.py x refcount = 4 y refcount = 4 

why 4 , not 2?.

when run pytest

$ python3 -m pytest stackoverflow_test.py =================== test session starts =================== platform linux -- python 3.4.3, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 rootdir: /opt/projects/0001_intomics/00005_textmining/jcr/textmining/tests, inifile: collected 2 items  stackoverflow_test.py ff  ======================== failures ========================= ___________________ test_ref_count_int ____________________      def test_ref_count_int():         x = 677461248192962146784178 >       assert sys.getrefcount(x) == 2 e       assert 3 == 2 e        +  3 = <built-in function getrefcount>(677461248192962146784178) e        +    <built-in function getrefcount> = sys.getrefcount  stackoverflow_test.py:7: assertionerror ___________________ test_ref_count_str ____________________      def test_ref_count_str():         y = 'very long , probbably unique string' >       assert sys.getrefcount(y) == 2 e       assertionerror: assert 3 == 2 e        +  3 = <built-in function getrefcount>('very long , probbably unique string') e        +    <built-in function getrefcount> = sys.getrefcount  stackoverflow_test.py:11: assertionerror 

why 3 , not 2?

question: how come that

• python = 4 ref counts
• pytest = 3 ref counts
• ipython session = 2 ref counts

i expect behave in ipython in 3 cases, can explain going on, , give me hints how best test objects creating.

literals in code stored in code object. bytecode stack reference:

>>> import dis >>> def normal_te_st(): ...     x = 222677461248192962146784178 ...     y = '!!!!very long , probbably unique string!!!!' ...     print ('x refcount = {}'.format(sys.getrefcount(x))) ...     print ('y refcount = {}'.format(sys.getrefcount(y))) ... >>> normal_te_st.__code__.co_consts (none, 222677461248192962146784178, '!!!!very long , probbably unique string!!!!', 'x refcount = {}', 'y refcount = {}') >>> dis.dis(normal_te_st)   2           0 load_const               1 (222677461248192962146784178)               2 store_fast               0 (x)    3           4 load_const               2 ('!!!!very long , probbably unique string!!!!')               6 store_fast               1 (y)    4           8 load_global              0 (print)              10 load_const               3 ('x refcount = {}')              12 load_attr                1 (format)              14 load_global              2 (sys)              16 load_attr                3 (getrefcount)              18 load_fast                0 (x)              20 call_function            1              22 call_function            1              24 call_function            1              26 pop_top    5          28 load_global              0 (print)              30 load_const               4 ('y refcount = {}')              32 load_attr                1 (format)              34 load_global              2 (sys)              36 load_attr                3 (getrefcount)              38 load_fast                1 (y)              40 call_function            1              42 call_function            1              44 call_function            1              46 pop_top              48 load_const               0 (none)              50 return_value 

the load_const opcodes load object co_consts tuple attached code object; tuple 1 reference. store_fast puts local variable, that's second reference.

then there's load_fast opcode, takes name local storage , puts on stack, again incrementing reference count.

last not least, pass value sys.getrefcount() call.

if want learn references objects, may want @ gc.get_referrers(); function excludes , stack when called, can mentally add +2:

>>> import gc >>> def gc_demo(): ...     x = 222677461248192962146784178 ...     print(gc.get_referrers(x)) ... >>> gc_demo() [(none, 222677461248192962146784178), <frame object @ 0x106a25a98>] 

that prints 2 objects; co_consts tuple, , current call frame (for locals).

py.test additional import-time magic rewrites assert statements, , result reference count different again.

you may want read reference counts section of extending python c or c++ documentation, objects, types , reference counts section of c api reference manual, , last not least debugging builds section of same same, learn how create python build helps trace reference counts in detail.

you should never rely on specific number of references object. can trivially add more references objects reaching function object, example (foo = normal_te_st.__code__.co_conts[1] increment reference count before running function). requires reference count go implementation detail. make sure your own code handles references correctly.