If You Create an Objects Thats a Pointer and Then Set It to New Again What Happens
Watch Now This tutorial has a related video grade created by the Real Python team. Watch information technology together with the written tutorial to deepen your understanding: Pointers and Objects in Python
If you've ever worked with lower level languages like C or C++, and then you've probably heard of pointers. Pointers let you to create great efficiency in parts of your lawmaking. They also crusade defoliation for beginners and can lead to various retentiveness direction bugs, even for experts. So where are they in Python, and how can you simulate pointers in Python?
Pointers are widely used in C and C++. Essentially, they are variables that concur the memory accost of another variable. For a refresher on pointers, y'all might consider checking out this overview on C Pointers.
In this commodity, you'll gain a improve understanding of Python'southward object model and learn why pointers in Python don't really be. For the cases where y'all demand to mimic arrow behavior, you'll learn ways to simulate pointers in Python without the retention-direction nightmare.
In this article, you'll:
- Learn why pointers in Python don't exist
- Explore the departure between C variables and Python names
- Simulate pointers in Python
- Experiment with existent pointers using
ctypes
Why Doesn't Python Have Pointers?
The truth is that I don't know. Could pointers in Python exist natively? Probably, just pointers seem to go against the Zen of Python. Pointers encourage implicit changes rather than explicit. Often, they are complex instead of simple, specially for beginners. Even worse, they beg for means to shoot yourself in the human foot, or do something actually dangerous like read from a section of memory yous were not supposed to.
Python tends to endeavor to abstract away implementation details like retentivity addresses from its users. Python often focuses on usability instead of speed. As a result, pointers in Python don't actually make sense. Not to fear though, Python does, by default, give you some of the benefits of using pointers.
Understanding pointers in Python requires a short detour into Python's implementation details. Specifically, you lot'll need to understand:
- Immutable vs mutable objects
- Python variables/names
Hold onto your memory addresses, and permit's get started.
Objects in Python
In Python, everything is an object. For proof, y'all can open a REPL and explore using isinstance():
>>>
>>> isinstance ( 1 , object ) True >>> isinstance ( list (), object ) True >>> isinstance ( True , object ) True >>> def foo (): ... laissez passer ... >>> isinstance ( foo , object ) True This code shows you that everything in Python is indeed an object. Each object contains at least 3 pieces of information:
- Reference count
- Type
- Value
The reference count is for memory management. For an in-depth look at the internals of retentiveness management in Python, you lot can read Retentiveness Management in Python.
The type is used at the CPython layer to ensure blazon safety during runtime. Finally, there'south the value, which is the actual value associated with the object.
Not all objects are the same though. There is 1 other important stardom you'll demand to sympathise: immutable vs mutable objects. Understanding the difference between the types of objects actually helps clarify the get-go layer of the onion that is pointers in Python.
Immutable vs Mutable Objects
In Python, there are two types of objects:
- Immutable objects can't be changed.
- Mutable objects can be changed.
Understanding this deviation is the beginning key to navigating the landscape of pointers in Python. Hither'southward a breakdown of common types and whether or not they are mutable or immutable:
| Type | Immutable? |
|---|---|
int | Yes |
float | Yeah |
bool | Yes |
complex | Yes |
tuple | Aye |
frozenset | Yes |
str | Yes |
listing | No |
set up | No |
dict | No |
Every bit you can encounter, lots of ordinarily used primitive types are immutable. You tin can prove this yourself by writing some Python. You lot'll demand a couple of tools from the Python standard library:
-
id()returns the object'due south retentivity address. -
isreturnsTrueif and only if two objects take the same memory address.
Once again, yous can utilize these in a REPL environment:
>>>
>>> ten = five >>> id ( x ) 94529957049376 In the above code, you have assigned the value 5 to x. If you tried to change this value with addition, then you lot'd get a new object:
>>>
>>> x += 1 >>> x six >>> id ( 10 ) 94529957049408 Even though the above code appears to modify the value of ten, yous're getting a new object equally a response.
The str type is also immutable:
>>>
>>> s = "real_python" >>> id ( s ) 140637819584048 >>> s += "_rocks" >>> due south 'real_python_rocks' >>> id ( southward ) 140637819609424 Again, south ends upward with a different memory addresses later on the += functioning.
Trying to directly mutate the string due south results in an error:
>>>
>>> s [ 0 ] = "R" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'str' object does not support item assignment The above lawmaking fails, and Python indicates that str doesn't support this mutation, which is in line with the definition that the str type is immutable.
Contrast that with a mutable object, like listing:
>>>
>>> my_list = [ 1 , 2 , iii ] >>> id ( my_list ) 140637819575368 >>> my_list . append ( 4 ) >>> my_list [1, ii, three, iv] >>> id ( my_list ) 140637819575368 This code shows a major difference in the two types of objects. my_list has an id originally. Even after four is appended to the list, my_list has the same id. This is considering the list blazon is mutable.
Another manner to demonstrate that the list is mutable is with assignment:
>>>
>>> my_list [ 0 ] = 0 >>> my_list [0, 2, three, four] >>> id ( my_list ) 140637819575368 In this code, you mutate my_list and set its kickoff element to 0. However, it maintains the aforementioned id even after this assignment. With mutable and immutable objects out of the way, the adjacent step on your journey to Python enlightenment is understanding Python'southward variable ecosystem.
Understanding Variables
Python variables are fundamentally unlike than variables in C or C++. In fact, Python doesn't even have variables. Python has names, non variables.
This might seem pedantic, and for the most part, information technology is. Near of the time, it's perfectly acceptable to think about Python names as variables, but agreement the deviation is of import. This is peculiarly true when you're navigating the tricky discipline of pointers in Python.
To help bulldoze home the difference, you can have a look at how variables work in C, what they represent, and then contrast that with how names work in Python.
Variables in C
Let'south say you had the following lawmaking that defines the variable x:
This 1 line of code has several, distinct steps when executed:
- Classify enough memory for an integer
- Assign the value
2337to that memory location - Indicate that
xpoints to that value
Shown in a simplified view of memory, it might expect like this:
Here, you can encounter that the variable x has a fake memory location of 0x7f1 and the value 2337. If, subsequently in the program, you want to change the value of x, you tin do the following:
The above code assigns a new value (2338) to the variable 10, thereby overwriting the previous value. This ways that the variable x is mutable. The updated memory layout shows the new value:
Notice that the location of 10 didn't alter, but the value itself. This is a pregnant indicate. It ways that ten is the retentiveness location, non simply a proper name for it.
Another way to think of this concept is in terms of buying. In one sense, x owns the retention location. x is, at outset, an empty box that tin fit exactly one integer in which integer values tin can be stored.
When you assign a value to x, you're placing a value in the box that x owns. If you wanted to introduce a new variable (y), you could add this line of lawmaking:
This lawmaking creates a new box called y and copies the value from 10 into the box. Now the memory layout will await similar this:
Observe the new location 0x7f5 of y. Even though the value of ten was copied to y, the variable y owns some new address in memory. Therefore, you could overwrite the value of y without affecting x:
At present the memory layout will wait like this:
Again, you lot accept modified the value at y, but not its location. In addition, y'all have not afflicted the original x variable at all. This is in stark dissimilarity with how Python names work.
Names in Python
Python does not have variables. It has names. Aye, this is a pedantic point, and you can certainly use the term variables as much as yous like. It is of import to know that there is a divergence between variables and names.
Permit's have the equivalent code from the above C example and write information technology in Python:
Much similar in C, the higher up lawmaking is broken downwardly into several singled-out steps during execution:
- Create a
PyObject - Set the typecode to integer for the
PyObject - Set the value to
2337for thePyObject - Create a name called
10 - Point
tento the newPyObject - Increase the refcount of the
PyObjectpast 1
In retentiveness, it might looks something like this:
You can meet that the memory layout is vastly dissimilar than the C layout from before. Instead of x owning the block of memory where the value 2337 resides, the newly created Python object owns the memory where 2337 lives. The Python proper name x doesn't straight own any memory accost in the manner the C variable 10 owned a static slot in retention.
If you were to attempt to assign a new value to x, you could try the post-obit:
What's happening hither is dissimilar than the C equivalent, but not too unlike from the original bind in Python.
This code:
- Creates a new
PyObject - Sets the typecode to integer for the
PyObject - Sets the value to
2338for thePyObject - Points
tento the newPyObject - Increases the refcount of the new
PyObjectby 1 - Decreases the refcount of the old
PyObjectpast 1
Now in retention, it would look something like this:
This diagram helps illustrate that ten points to a reference to an object and doesn't own the retentivity space as before. It also shows that the x = 2338 command is not an assignment, merely rather binding the name x to a reference.
In addition, the previous object (which held the 2337 value) is at present sitting in memory with a ref count of 0 and volition get cleaned up past the garbage collector.
You could introduce a new name, y, to the mix as in the C example:
In memory, you would have a new name, but non necessarily a new object:
Now you tin can run into that a new Python object has not been created, merely a new proper noun that points to the same object. Also, the object'south refcount has increased past one. You could bank check for object identity equality to confirm that they are the same:
The above code indicates that x and y are the aforementioned object. Make no mistake though: y is still immutable.
For example, you lot could perform addition on y:
>>>
>>> y += 1 >>> y is x False Subsequently the addition call, y'all are returned with a new Python object. Now, the memory looks similar this:
A new object has been created, and y now points to the new object. Interestingly, this is the same stop-state if y'all had bound y to 2339 directly:
The higher up statement results in the same end-memory state equally the improver. To epitomize, in Python, you don't assign variables. Instead, you bind names to references.
A Note on Intern Objects in Python
At present that you understand how Python objects get created and names get bound to those objects, its time to throw a wrench in the machinery. That wrench goes by the name of interned objects.
Suppose you have the following Python code:
>>>
>>> x = 1000 >>> y = 1000 >>> x is y True As above, x and y are both names that betoken to the same Python object. Simply the Python object that holds the value 1000 is not always guaranteed to have the same memory accost. For case, if you were to add two numbers together to go yard, you lot would end upward with a dissimilar memory address:
>>>
>>> x = 1000 >>> y = 499 + 501 >>> ten is y False This fourth dimension, the line x is y returns Imitation. If this is confusing, then don't worry. Hither are the steps that occur when this lawmaking is executed:
- Create Python object(
thousand) - Assign the name
xto that object - Create Python object (
499) - Create Python object (
501) - Add these two objects together
- Create a new Python object (
chiliad) - Assign the proper name
yto that object
Isn't this wasteful? Well, aye it is, just that's the price you lot pay for all of the great benefits of Python. You never have to worry almost cleaning up these intermediate objects or even need to know that they be! The joy is that these operations are relatively fast, and you never had to know any of those details until at present.
The core Python developers, in their wisdom, likewise noticed this waste and decided to make a few optimizations. These optimizations outcome in behavior that can be surprising to newcomers:
>>>
>>> x = xx >>> y = 19 + 1 >>> x is y True In this example, you see nearly the aforementioned lawmaking as before, except this time the event is Truthful. This is the consequence of interned objects. Python pre-creates a sure subset of objects in retentiveness and keeps them in the global namespace for everyday use.
Which objects depend on the implementation of Python. CPython 3.vii interns the following:
- Integer numbers between
-5and256 - Strings that incorporate ASCII messages, digits, or underscores only
The reasoning backside this is that these variables are extremely likely to be used in many programs. By interning these objects, Python prevents memory allocation calls for consistently used objects.
Strings that are less than xx characters and contain ASCII messages, digits, or underscores volition exist interned. The reasoning behind this is that these are assumed to be some kind of identity:
>>>
>>> s1 = "realpython" >>> id ( s1 ) 140696485006960 >>> s2 = "realpython" >>> id ( s2 ) 140696485006960 >>> s1 is s2 True Hither you can see that s1 and s2 both signal to the same accost in memory. If you were to introduce a non-ASCII letter, digit, or underscore, then yous would get a different result:
>>>
>>> s1 = "Real Python!" >>> s2 = "Real Python!" >>> s1 is s2 False Because this example has an assertion mark (!) in it, these strings are not interned and are different objects in retentivity.
Interned objects are frequently a source of confusion. Just remember, if you're ever in doubtfulness, that you lot can always apply id() and is to make up one's mind object equality.
Simulating Pointers in Python
But because pointers in Python don't exist natively doesn't mean you can't become the benefits of using pointers. In fact, in that location are multiple ways to simulate pointers in Python. Y'all'll acquire two in this section:
- Using mutable types as pointers
- Using custom Python objects
Okay, allow's go to the signal.
Using Mutable Types as Pointers
You've already learned about mutable types. Because these objects are mutable, you tin treat them as if they were pointers to simulate pointer beliefs. Suppose yous wanted to replicate the post-obit c code:
void add_one ( int * 10 ) { * x += 1 ; } This code takes a pointer to an integer (*ten) and so increments the value past 1. Here is a main function to exercise the code:
#include <stdio.h> int main ( void ) { int y = 2337 ; printf ( "y = %d \n " , y ); add_one ( & y ); printf ( "y = %d \n " , y ); return 0 ; } In the to a higher place code, you assign 2337 to y, print out the current value, increase the value by one, and and so print out the modified value. The output of executing this lawmaking would be the following:
One way to replicate this type of behavior in Python is past using a mutable type. Consider using a list and modifying the first element:
>>>
>>> def add_one ( x ): ... x [ 0 ] += 1 ... >>> y = [ 2337 ] >>> add_one ( y ) >>> y [ 0 ] 2338 Here, add_one(10) accesses the beginning element and increments its value by 1. Using a list ways that the end effect appears to take modified the value. And then pointers in Python do exist? Well, no. This is only possible because list is a mutable type. If you lot tried to use a tuple, you would become an error:
>>>
>>> z = ( 2337 ,) >>> add_one ( z ) Traceback (well-nigh contempo call terminal): File "<stdin>", line i, in <module> File "<stdin>", line two, in add_one TypeError: 'tuple' object does non support item assignment The higher up code demonstrates that tuple is immutable. Therefore, it does non support particular consignment. listing is non the merely mutable type. Some other mutual approach to mimicking pointers in Python is to utilise a dict.
Let's say you had an application where you wanted to keep track of every time an interesting event happened. One manner to achieve this would be to create a dict and use 1 of the items as a counter:
>>>
>>> counters = { "func_calls" : 0 } >>> def bar (): ... counters [ "func_calls" ] += 1 ... >>> def foo (): ... counters [ "func_calls" ] += i ... bar () ... >>> foo () >>> counters [ "func_calls" ] two In this case, the counters lexicon is used to go along runway of the number of function calls. After you lot call foo(), the counter has increased to 2 as expected. All because dict is mutable.
Keep in heed, this is only simulates arrow behavior and does non directly map to true pointers in C or C++. That is to say, these operations are more than expensive than they would be in C or C++.
Using Python Objects
The dict option is a great way to emulate pointers in Python, but sometimes it gets tedious to remember the key proper noun yous used. This is peculiarly true if you're using the lexicon in various parts of your application. This is where a custom Python form can really help.
To build on the last example, assume that you want to runway metrics in your application. Creating a form is a slap-up way to abstract the pesky details:
form Metrics ( object ): def __init__ ( self ): cocky . _metrics = { "func_calls" : 0 , "cat_pictures_served" : 0 , } This code defines a Metrics course. This class still uses a dict for holding the actual information, which is in the _metrics member variable. This will give you the mutability yous need. Now y'all just need to be able to admission these values. One nice way to exercise this is with properties:
grade Metrics ( object ): # ... @property def func_calls ( self ): return self . _metrics [ "func_calls" ] @holding def cat_pictures_served ( cocky ): return cocky . _metrics [ "cat_pictures_served" ] This code makes use of @holding. If y'all're not familiar with decorators, you tin can check out this Primer on Python Decorators. The @property decorator here allows you lot to admission func_calls and cat_pictures_served as if they were attributes:
>>>
>>> metrics = Metrics () >>> metrics . func_calls 0 >>> metrics . cat_pictures_served 0 The fact that you tin can access these names as attributes means that you abstracted the fact that these values are in a dict. You also make it more explicit what the names of the attributes are. Of course, y'all need to exist able to increase these values:
class Metrics ( object ): # ... def inc_func_calls ( cocky ): cocky . _metrics [ "func_calls" ] += ane def inc_cat_pics ( self ): self . _metrics [ "cat_pictures_served" ] += one You accept introduced two new methods:
-
inc_func_calls() -
inc_cat_pics()
These methods change the values in the metrics dict. Y'all now have a form that you change as if you lot're modifying a arrow:
>>>
>>> metrics = Metrics () >>> metrics . inc_func_calls () >>> metrics . inc_func_calls () >>> metrics . func_calls 2 Here, y'all tin access func_calls and call inc_func_calls() in various places in your applications and simulate pointers in Python. This is useful when you have something like metrics that demand to be used and updated ofttimes in diverse parts of your applications.
Here'southward the total source for the Metrics course:
class Metrics ( object ): def __init__ ( self ): cocky . _metrics = { "func_calls" : 0 , "cat_pictures_served" : 0 , } @belongings def func_calls ( self ): return self . _metrics [ "func_calls" ] @property def cat_pictures_served ( self ): render self . _metrics [ "cat_pictures_served" ] def inc_func_calls ( self ): cocky . _metrics [ "func_calls" ] += 1 def inc_cat_pics ( self ): self . _metrics [ "cat_pictures_served" ] += 1 Real Pointers With ctypes
Okay, and so perhaps there are pointers in Python, specifically CPython. Using the builtin ctypes module, you can create real C-fashion pointers in Python. If you are unfamiliar with ctypes, then you lot can take a await at Extending Python With C Libraries and the "ctypes" Module.
The real reason you would use this is if you needed to make a office call to a C library that requires a pointer. Let'south go dorsum to the add_one() C-function from earlier:
void add_one ( int * 10 ) { * x += 1 ; } Here again, this lawmaking is incrementing the value of 10 by 1. To employ this, first compile information technology into a shared object. Assuming the above file is stored in add together.c, you lot could accomplish this with gcc:
$ gcc -c -Wall -Werror -fpic add.c $ gcc -shared -o libadd1.so add.o The first control compiles the C source file into an object called add.o. The second control takes that unlinked object file and produces a shared object called libadd1.so.
libadd1.so should be in your current directory. You tin load information technology into Python using ctypes:
>>>
>>> import ctypes >>> add_lib = ctypes . CDLL ( "./libadd1.so" ) >>> add_lib . add_one <_FuncPtr object at 0x7f9f3b8852a0> The ctypes.CDLL code returns an object that represents the libadd1 shared object. Because yous defined add_one() in this shared object, you can access it equally if it were any other Python object. Before y'all call the office though, you should specify the function signature. This helps Python ensure that you pass the right type to the function.
In this case, the role signature is a pointer to an integer. ctypes will permit you to specify this using the following code:
>>>
>>> add_one = add_lib . add_one >>> add_one . argtypes = [ ctypes . POINTER ( ctypes . c_int )] In this lawmaking, you're setting the function signature to match what C is expecting. At present, if you lot were to attempt to call this code with the wrong type, then you would get a nice warning instead of undefined beliefs:
>>>
>>> add_one ( 1 ) Traceback (most recent call last): File "<stdin>", line 1, in <module> ctypes.ArgumentError: argument one: <grade 'TypeError'>: \ expected LP_c_int example instead of int Python throws an mistake, explaining that add_one() wants a pointer instead of merely an integer. Luckily, ctypes has a manner to pass pointers to these functions. Beginning, declare a C-fashion integer:
>>>
>>> x = ctypes . c_int () >>> x c_int(0) The in a higher place code creates a C-style integer ten with a value of 0. ctypes provides the handy byref() to permit passing a variable by reference.
You can use this to call add_one():
>>>
>>> add_one ( ctypes . byref ( x )) 998793640 >>> x c_int(1) Nice! Your integer was incremented by one. Congratulations, you have successfully used existent pointers in Python.
Conclusion
You now take a better agreement of the intersection between Python objects and pointers. Even though some of the distinctions between names and variables seem pedantic, fundamentally understanding these central terms expands your understanding of how Python handles variables.
You've besides learned some fantabulous ways to simulate pointers in Python:
- Utilizing mutable objects every bit low-overhead pointers
- Creating custom Python objects for ease of apply
- Unlocking existent pointers with the
ctypesmodule
These methods allow you to simulate pointers in Python without sacrificing the retentivity safety that Python provides.
Thanks for reading. If you still have questions, feel complimentary to accomplish out either in the comments section or on Twitter.
Sentry Now This tutorial has a related video course created past the Real Python squad. Watch information technology together with the written tutorial to deepen your understanding: Pointers and Objects in Python
russelldesets1966.blogspot.com
Source: https://realpython.com/pointers-in-python/
0 Response to "If You Create an Objects Thats a Pointer and Then Set It to New Again What Happens"
Post a Comment