mutable/immutable

A forum for general discussion of the Python programming language.

mutable/immutable

Postby Mekire » Tue Feb 19, 2013 4:49 am

There was an interesting conversation before the death of the previous forum in which people were discussing mutable/immutable objects, and the underlying mechanics that are occurring. One of the common things people tend to say is, mutable types are passed by reference, but immutable types are passed by value. This is however an untrue statement with regards to Python; in Python everything is technically passed by reference.

Some interesting things I have read/encountered:

The first involves the immutable type int. Small integers of the same value will generally use the same reference even when you create a "new" one, but larger integers like to actually create new objects:
Code: Select all
>>> a = 7
>>> b = 7
>>> a == b
True
>>> a is b
True
>>> a = 10005
>>> b = 10005
>>> a == b
True
>>> a is b
False
The same is true of simple strings vs complex strings. Simple ones are assigned the same reference but complex ones aren't:
Code: Select all
>>> a = "artichoke"
>>> b = "artichoke"
>>> a==b
True
>>> a is b
True
>>> a = "It's the end of the world"
>>> b = "It's the end of the world"
>>> a==b
True
>>> a is b
False
Of course, as the object is immutable this will never affect anything.

The next thing regards mutable types, specifically lists, and as they are mutable this can be quite the gotcha. The simplest case of this often occurs when people try to initialize multiple variables at once:
Code: Select all
>>> a = b = []
>>> a.append("spam")
>>> b
['spam']
This may or may not have been the intention of the programmer, but nonetheless it is quite easy to avoid if you understand you are only creating one object.

The following case however is a bit more subtle and has bitten me previously:
Code: Select all
def myfunc(arg = []):
    arg += [1,2,3]
    return arg
a = myfunc()
b = myfunc()
print(a,a is b)
With result:
Code: Select all
([1, 2, 3, 1, 2, 3], True)
the value of a changed twice; in fact a and b are the same object. The following is much safer:
Code: Select all
def myfunc(arg = None):
    if not arg: arg = []
    arg += [1,2,3]
    return arg
a = myfunc()
b = myfunc()
print(a,a is b)
Now a does not change on the second call. The lesson I learned here was that you should never try to use a basic-mutable as a default argument. In fact if you are even passing a mutable to a function it would be best practice to pass a copy rather than the original object unless you expect the function to perform an in-place change.

Anyone have anything interesting to say on the matter? (pardon my long post)
-Mek
User avatar
Mekire
 
Posts: 814
Joined: Thu Feb 07, 2013 11:33 pm
Location: Amakusa, Japan

Re: mutable/immutable

Postby lovecodecakes » Tue Feb 19, 2013 9:49 am

don't know about "is" syntax. but address & id() in both cases as a==b & a is b could make the difference.true address & id aren't the same.i've checked that in a code to match [0] & [-1] of a >2 len word
what happened was if: gaaaag or gibberish were arg, the 1st one had id check and [0] & [-1] had same id.cant really be in the memory register level.it is all physically stored. so id is not true address.probably that is why here in a is b, is fails. in the second word gibberish,say, since g & h are not same char/entity, it had to use a diff id. I feel python really knows how to save memory on that part. (your remarks??)
other than that, since python is a layer above c, I feel pointers is all there is to ref each object like a=something and b=a.efficient way of saving memory.

I think the first code returns arg,a,a is b in the same list[] coz a & b are pointing to the same thing and interpreter automatically wraps the whole memory segment..continuous segment x99..x00....xFF since there is no other variable & code storing any other value "in-between".I think if u try out that,if there were other values being saved, that wrapping of return arg and print a,ais b wont be given twice in the same line.what say?
lovecodecakes
 
Posts: 56
Joined: Mon Feb 11, 2013 8:19 pm

Re: mutable/immutable

Postby Mekire » Tue Feb 19, 2013 11:21 am

id() does, on cPython at the very least, return a memory address.
The reason that a string like this:
Code: Select all
mystring = "ghargaaaaaaaag"
print(mystring[0] == mystring[-1])
print(mystring[0] is mystring[-1])
print(hex(id(mystring[0])),hex(id(mystring[-1])))
Code: Select all
True
True
('0x1c7d4d0', '0x1c7d4d0')
says the first and last items are at the same memory address, is because they are.

The misconception here is that mystring is a pointer to an array of characters and that the first and last item therefore can't be at the same place. It is not. It is a pointer to an array of pointers to characters. In python absolutely everything is a pointer/reference. Someone please correct me if I am mistaken.

The pointer in position [0] and the pointer in position [-1] point to the same place.
(It is also possible I didn't understand what you were trying to say. Please consider formatting your responses a little more clearly)

-Mek
User avatar
Mekire
 
Posts: 814
Joined: Thu Feb 07, 2013 11:33 pm
Location: Amakusa, Japan

Re: mutable/immutable

Postby lovecodecakes » Tue Feb 19, 2013 1:50 pm

The way I understand what you said :
a pointer to an array of pointers to characters
mystring = "ghargaaaaaaaag"
==> *(char arr[])=*(*ptr1,*ptr2...*ptrn) ??
where ptrn=char array_n[] ??

Ghastly.

I just knew char *name="whatever you say will be held here with \spaces included so I use this technique"

I meant just what I wrote.
cant really be in the memory register level.it is all physically stored. so id() is not true address.id() is checking for similar ascii values to point to.
probably that is why here in a is b, is fails. in the second word gibberish,say, since g & h are not same char/entity, it had to use a diff id().
pointers is all there is to ref each object like a=something and b=a.

And if there is memory segmentation,interpreter can take successive memory and make a big list if no one tells it to create a different list using a different list object.
lovecodecakes
 
Posts: 56
Joined: Mon Feb 11, 2013 8:19 pm

Re: mutable/immutable

Postby lovecodecakes » Tue Feb 19, 2013 1:56 pm

Mekire wrote: Someone please correct me if I am mistaken.

The pointer in position [0] and the pointer in position [-1] point to the same place.

-Mek


You are right when talkin abt ref. but its still confusin why is in a is b doesnt do that.
according to this code:

Code: Select all
static PyObject *
 builtin_id(PyObject *self, PyObject *v)
{
     return PyLong_FromVoidPtr(v);
}


returns value that is the address of the object in the memory. but then the smart interpreter chks for similarily in list indexes..or tuples indexes..similar values in id()s become same address.
From here>> http://tharindu-rusira.blogspot.com/2012/09/python-and-pointers.htmlthe author has also noted(when u read d whole article) that:
"One thing is clear from the simple tests above, this id(Object) does NOT give me the pointer values in the sense it is used in C/C++."
Last edited by lovecodecakes on Tue Feb 19, 2013 2:05 pm, edited 1 time in total.
lovecodecakes
 
Posts: 56
Joined: Mon Feb 11, 2013 8:19 pm

Re: mutable/immutable

Postby Mekire » Tue Feb 19, 2013 2:56 pm

The author of that article is from a very pedantic C background as he admits.
He is also a fairly novice python programmer.
He is used to being able to directly manipulate the data that exists at a memory address.
This cannot be done through conventional means in python.

Also because he is accustomed to directly manipulating memory it disturbs him that different variables would point to the same object in memory. This is confusion on the point of mutable vs immutable, which is exactly what I initially brought up. The fact that small integers and small strings may point to the same object will never cause issues because the values at those memory addresses can not be changed.

He does not claim that id() does not return a memory address. In my opinion, his complaint or discovery is rather, that even knowing the memory address he can not manipulate it as he does in C.

I personally have not studied C in many years now and honestly I always hated it (as I was introduced to C and python simultaneously). I would love to explicitly prove that id() is indeed returning a memory address; unfortunately I don't know how I would go about that. Perhaps someone else does?

-Mek
User avatar
Mekire
 
Posts: 814
Joined: Thu Feb 07, 2013 11:33 pm
Location: Amakusa, Japan

Re: mutable/immutable

Postby lovecodecakes » Tue Feb 19, 2013 3:41 pm

Interesting mek, as I share the same python enthu here, being a C coder for a while.u can see my C approach even in my coding logic. the difficulty of low level address & memory in python & im still discovering as Im a python novice. Well anyone would answer that question provided they know the internal variable treatment.look at this : http://stackoverflow.com/questions/9860 ... -reference

UPDATE: http://rosettacode.org/wiki/Pointers_an ... ces#Python
I think this Is worth noting.
"Python does not have pointers"
lovecodecakes
 
Posts: 56
Joined: Mon Feb 11, 2013 8:19 pm

Re: mutable/immutable

Postby casevh » Tue Feb 19, 2013 4:33 pm

Python doesn't expose pointers. When an object is created in Python, a structure is created in memory that contains information about the object. In addition to the actual data (the value of an integer or a sequence of bytes that can be interpreted as a string), a reference count field is present. I'll skip over some of the details but this is a reasonable summary. The following code:

Code: Select all
a = foo()


will create an object of type foo and set the reference count of the object to 1. a now is a reference to that object. When you do:

Code: Select all
b = a


the reference count of the object is increased to 2 and both a and b are references to the same object.

When you do:

Code: Select all
a += 1


the behavior depends on the capabilities of the type foo. If foo mimics a normal integer type, then the following sequence is followed:

1) a new object is created with a reference count of 1
2) it is given the same value as the original object
3) the value is increased by one
4) a is assigned the new object
4.1) the reference count of the original object is decreased by 1
4.2) a now refers to the second object which has a reference count of 1

Whether an object is mutable or immutable is determined entirely by the functions/methods that are defined by the object's type. For mutable objects, the functions/methods modify the object itself, regardless of how many references are pointing to it. For immutable objects, the functions/methods always created new objects with a different value instead of modifying the original object.
casevh
 
Posts: 56
Joined: Sat Feb 09, 2013 7:35 am


Return to General Discussions

Who is online

Users browsing this forum: Google [Bot] and 1 guest