Working with Cookies¶
Getting started¶
The cookies mapping has an extended mapping interface that allows getting, setting, and deleting the cookies that the browser is remembering for the current url, or for an explicitly provided URL.
>>> from zope.testbrowser.ftests.wsgitestapp import WSGITestApplication
>>> from zope.testbrowser.wsgi import Browser
>>> wsgi_app = WSGITestApplication()
>>> browser = Browser(wsgi_app=wsgi_app)
Initially the browser does not point to a URL, and the cookies cannot be used.
>>> len(browser.cookies)
Traceback (most recent call last):
...
RuntimeError: no request found
>>> browser.cookies.keys()
Traceback (most recent call last):
...
RuntimeError: no request found
Once you send the browser to a URL, the cookies attribute can be used.
>>> browser.open('http://localhost/@@/testbrowser/simple.html')
>>> len(browser.cookies)
0
>>> browser.cookies.keys()
[]
>>> browser.url
'http://localhost/@@/testbrowser/simple.html'
>>> browser.cookies.url
'http://localhost/@@/testbrowser/simple.html'
>>> import zope.testbrowser.interfaces
>>> from zope.interface.verify import verifyObject
>>> verifyObject(zope.testbrowser.interfaces.ICookies, browser.cookies)
True
Alternatively, you can use the forURL
method to get another instance of
the cookies mapping for the given URL.
>>> len(browser.cookies.forURL('http://www.example.com'))
0
>>> browser.cookies.forURL('http://www.example.com').keys()
[]
>>> browser.cookies.forURL('http://www.example.com').url
'http://www.example.com'
>>> browser.url
'http://localhost/@@/testbrowser/simple.html'
>>> browser.cookies.url
'http://localhost/@@/testbrowser/simple.html'
Here, we use a view that will make the server set cookies with the values we provide.
>>> browser.open('http://localhost/set_cookie.html?name=foo&value=bar')
>>> browser.headers['set-cookie'].replace(';', '')
'foo=bar'
Basic Mapping Interface¶
Now the cookies for localhost have a value. These are examples of just the basic accessor operators and methods.
>>> browser.cookies['foo']
'bar'
>>> list(browser.cookies.keys())
['foo']
>>> list(browser.cookies.values())
['bar']
>>> list(browser.cookies.items())
[('foo', 'bar')]
>>> 'foo' in browser.cookies
True
>>> 'bar' in browser.cookies
False
>>> len(browser.cookies)
1
>>> print(dict(browser.cookies))
{'foo': 'bar'}
As you would expect, the cookies attribute can also be used to examine cookies that have already been set in a previous request. To demonstrate this, we use another view that does not set cookies but reports on the cookies it receives from the browser.
>>> browser.open('http://localhost/get_cookie.html')
>>> print(browser.headers.get('set-cookie'))
None
>>> browser.contents
'foo: bar'
>>> browser.cookies['foo']
'bar'
The standard mapping mutation methods and operators are also available, as seen here.
>>> browser.cookies['sha'] = 'zam'
>>> len(browser.cookies)
2
>>> import pprint
>>> pprint.pprint(sorted(browser.cookies.items()))
[('foo', 'bar'), ('sha', 'zam')]
>>> browser.open('http://localhost/get_cookie.html')
>>> print(browser.headers.get('set-cookie'))
None
>>> print(browser.contents) # server got the cookie change
foo: bar
sha: zam
>>> browser.cookies.update({'va': 'voom', 'tweedle': 'dee'})
>>> pprint.pprint(sorted(browser.cookies.items()))
[('foo', 'bar'), ('sha', 'zam'), ('tweedle', 'dee'), ('va', 'voom')]
>>> browser.open('http://localhost/get_cookie.html')
>>> print(browser.headers.get('set-cookie'))
None
>>> print(browser.contents)
foo: bar
sha: zam
tweedle: dee
va: voom
>>> del browser.cookies['foo']
>>> del browser.cookies['tweedle']
>>> browser.open('http://localhost/get_cookie.html')
>>> print(browser.contents)
sha: zam
va: voom
Headers¶
You can see the Cookies header that will be sent to the browser in the
header
attribute and the repr and str.
>>> browser.cookies.header
'sha=zam; va=voom'
>>> browser.cookies
<zope.testbrowser.cookies.Cookies object at ... for
http://localhost/get_cookie.html (sha=zam; va=voom)>
>>> str(browser.cookies)
'sha=zam; va=voom'
Extended Mapping Interface¶
Read Methods: getinfo
and iterinfo
¶
getinfo
¶
The cookies
mapping also has an extended interface to get and set extra
information about each cookie.
zope.testbrowser.interfaces.ICookie.getinfo()
returns a dictionary.
Here are some examples.
>>> browser.open('http://localhost/set_cookie.html?name=foo&value=bar')
>>> pprint.pprint(browser.cookies.getinfo('foo'))
{'comment': None,
'commenturl': None,
'domain': 'localhost.local',
'expires': None,
'name': 'foo',
'path': '/',
'port': None,
'secure': False,
'value': 'bar'}
>>> pprint.pprint(browser.cookies.getinfo('sha'))
{'comment': None,
'commenturl': None,
'domain': 'localhost.local',
'expires': None,
'name': 'sha',
'path': '/',
'port': None,
'secure': False,
'value': 'zam'}
>>> import datetime
>>> expires = datetime.datetime(2030, 1, 1, 12, 22, 33).strftime(
... '%a, %d %b %Y %H:%M:%S GMT')
>>> browser.open(
... 'http://localhost/set_cookie.html?name=wow&value=wee&'
... 'expires=%s' %
... (expires,))
>>> pprint.pprint(browser.cookies.getinfo('wow'))
{'comment': None,
'commenturl': None,
'domain': 'localhost.local',
'expires': datetime.datetime(2030, 1, 1, 12, 22, ...tzinfo=<UTC>),
'name': 'wow',
'path': '/',
'port': None,
'secure': False,
'value': 'wee'}
Max-age is converted to an “expires” value.
>>> browser.open(
... 'http://localhost/set_cookie.html?name=max&value=min&'
... 'max-age=3000&&comment=silly+billy')
>>> pprint.pprint(browser.cookies.getinfo('max'))
{'comment': '"silly billy"',
'commenturl': None,
'domain': 'localhost.local',
'expires': datetime.datetime(..., tzinfo=<UTC>),
'name': 'max',
'path': '/',
'port': None,
'secure': False,
'value': 'min'}
iterinfo
¶
You can iterate over all of the information about the cookies for the current
page using the iterinfo
method.
>>> pprint.pprint(sorted(browser.cookies.iterinfo(),
... key=lambda info: info['name']))
...
[{'comment': None,
'commenturl': None,
'domain': 'localhost.local',
'expires': None,
'name': 'foo',
'path': '/',
'port': None,
'secure': False,
'value': 'bar'},
{'comment': '"silly billy"',
'commenturl': None,
'domain': 'localhost.local',
'expires': datetime.datetime(..., tzinfo=<UTC>),
'name': 'max',
'path': '/',
'port': None,
'secure': False,
'value': 'min'},
{'comment': None,
'commenturl': None,
'domain': 'localhost.local',
'expires': None,
'name': 'sha',
'path': '/',
'port': None,
'secure': False,
'value': 'zam'},
{'comment': None,
'commenturl': None,
'domain': 'localhost.local',
'expires': None,
'name': 'va',
'path': '/',
'port': None,
'secure': False,
'value': 'voom'},
{'comment': None,
'commenturl': None,
'domain': 'localhost.local',
'expires': datetime.datetime(2030, 1, 1, 12, 22, ...tzinfo=<UTC>),
'name': 'wow',
'path': '/',
'port': None,
'secure': False,
'value': 'wee'}]
Extended Examples¶
If you want to look at the cookies for another page, you can either navigate to
the other page in the browser, or, as already mentioned, you can use the
forURL
method, which returns an ICookies instance for the new URL.
>>> sorted(browser.cookies.forURL(
... 'http://localhost/inner/set_cookie.html').keys())
['foo', 'max', 'sha', 'va', 'wow']
>>> extra_cookie = browser.cookies.forURL(
... 'http://localhost/inner/set_cookie.html')
>>> extra_cookie['gew'] = 'gaw'
>>> extra_cookie.getinfo('gew')['path']
'/inner'
>>> sorted(extra_cookie.keys())
['foo', 'gew', 'max', 'sha', 'va', 'wow']
>>> sorted(browser.cookies.keys())
['foo', 'max', 'sha', 'va', 'wow']
>>> browser.open('http://localhost/inner/get_cookie.html')
>>> print(browser.contents) # has gewgaw
foo: bar
gew: gaw
max: min
sha: zam
va: voom
wow: wee
>>> browser.open('http://localhost/inner/path/get_cookie.html')
>>> print(browser.contents) # has gewgaw
foo: bar
gew: gaw
max: min
sha: zam
va: voom
wow: wee
>>> browser.open('http://localhost/get_cookie.html')
>>> print(browser.contents) # NO gewgaw
foo: bar
max: min
sha: zam
va: voom
wow: wee
Here’s an example of the server setting a cookie that is only available on an inner page.
>>> browser.open(
... 'http://localhost/inner/path/set_cookie.html?name=big&value=kahuna'
... )
>>> browser.cookies['big']
'kahuna'
>>> browser.cookies.getinfo('big')['path']
'/inner/path'
>>> browser.cookies.getinfo('gew')['path']
'/inner'
>>> browser.cookies.getinfo('foo')['path']
'/'
>>> print(browser.cookies.forURL('http://localhost/').get('big'))
None
Write Methods: create
and change
¶
The basic mapping API only allows setting values. If a cookie already exists for the given name, it’s value will be changed; or else a new cookie will be created for the current request’s domain and a path of ‘/’, set to last for only this browser session (a “session” cookie).
To create or change cookies with different additional information, use the
create
and change
methods, respectively. Here is an example of
create
.
>>> from pytz import UTC
>>> browser.cookies.create(
... 'bling', value='blang', path='/inner',
... expires=datetime.datetime(2030, 1, 1, tzinfo=UTC),
... comment='follow swallow')
>>> pprint.pprint(browser.cookies.getinfo('bling'))
{'comment': 'follow%20swallow',
'commenturl': None,
'domain': 'localhost.local',
'expires': datetime.datetime(2030, 1, 1, 0, 0, tzinfo=<UTC>),
'name': 'bling',
'path': '/inner',
'port': None,
'secure': False,
'value': 'blang'}
In these further examples of create
, note that the testbrowser sends all
domains to Zope, and both http and https.
>>> browser.open('https://dev.example.com/inner/path/get_cookie.html')
>>> browser.cookies.keys() # a different domain
[]
>>> browser.cookies.create('tweedle', 'dee')
>>> pprint.pprint(browser.cookies.getinfo('tweedle'))
{'comment': None,
'commenturl': None,
'domain': 'dev.example.com',
'expires': None,
'name': 'tweedle',
'path': '/inner/path',
'port': None,
'secure': False,
'value': 'dee'}
>>> browser.cookies.create(
... 'boo', 'yah', domain='.example.com', path='/inner', secure=True)
>>> pprint.pprint(browser.cookies.getinfo('boo'))
{'comment': None,
'commenturl': None,
'domain': '.example.com',
'expires': None,
'name': 'boo',
'path': '/inner',
'port': None,
'secure': True,
'value': 'yah'}
>>> sorted(browser.cookies.keys())
['boo', 'tweedle']
>>> browser.open('https://dev.example.com/inner/path/get_cookie.html')
>>> print(browser.contents)
boo: yah
tweedle: dee
>>> browser.open( # not https, so not secure, so not 'boo'
... 'http://dev.example.com/inner/path/get_cookie.html')
>>> sorted(browser.cookies.keys())
['tweedle']
>>> print(browser.contents)
tweedle: dee
>>> browser.open( # not tweedle's domain
... 'https://prod.example.com/inner/path/get_cookie.html')
>>> sorted(browser.cookies.keys())
['boo']
>>> print(browser.contents)
boo: yah
>>> browser.open( # not tweedle's domain
... 'https://example.com/inner/path/get_cookie.html')
>>> sorted(browser.cookies.keys())
['boo']
>>> print(browser.contents)
boo: yah
>>> browser.open( # not tweedle's path
... 'https://dev.example.com/inner/get_cookie.html')
>>> sorted(browser.cookies.keys())
['boo']
>>> print(browser.contents)
boo: yah
Masking by Path¶
The API allows creation of cookies that mask existing cookies, but it does not allow creating a cookie that will be immediately masked upon creation. Having multiple cookies with the same name for a given URL is rare, and is a pathological case for using a mapping API to work with cookies, but it is supported to some degree, as demonstrated below. Note that the Cookie RFCs (2109, 2965) specify that all matching cookies be sent to the server, but with an ordering so that more specific paths come first. We also prefer more specific domains, though the RFCs state that the ordering of cookies with the same path is indeterminate. The best-matching cookie is the one that the mapping API uses.
Also note that ports, as sent by RFC 2965’s Cookie2 and Set-Cookie2 headers, are parsed and stored by this API but are not used for filtering as of this writing.
This is an example of making one cookie that masks another because of path. First, unless you pass an explicit path, you will be modifying the existing cookie.
>>> browser.open('https://dev.example.com/inner/path/get_cookie.html')
>>> print(browser.contents)
boo: yah
tweedle: dee
>>> browser.cookies.getinfo('boo')['path']
'/inner'
>>> browser.cookies['boo'] = 'hoo'
>>> browser.cookies.getinfo('boo')['path']
'/inner'
>>> browser.cookies.getinfo('boo')['secure']
True
Now we mask the cookie, using the path.
>>> browser.cookies.create('boo', 'boo', path='/inner/path')
>>> browser.cookies['boo']
'boo'
>>> browser.cookies.getinfo('boo')['path']
'/inner/path'
>>> browser.cookies.getinfo('boo')['secure']
False
>>> browser.cookies['boo']
'boo'
>>> sorted(browser.cookies.keys())
['boo', 'tweedle']
To identify the additional cookies, you can change the URL…
>>> extra_cookies = browser.cookies.forURL(
... 'https://dev.example.com/inner/get_cookie.html')
>>> extra_cookies['boo']
'hoo'
>>> extra_cookies.getinfo('boo')['path']
'/inner'
>>> extra_cookies.getinfo('boo')['secure']
True
…or use iterinfo
and pass in a name.
>>> pprint.pprint(list(browser.cookies.iterinfo('boo')))
[{'comment': None,
'commenturl': None,
'domain': 'dev.example.com',
'expires': None,
'name': 'boo',
'path': '/inner/path',
'port': None,
'secure': False,
'value': 'boo'},
{'comment': None,
'commenturl': None,
'domain': '.example.com',
'expires': None,
'name': 'boo',
'path': '/inner',
'port': None,
'secure': True,
'value': 'hoo'}]
An odd situation in this case is that deleting a cookie can sometimes reveal another one.
>>> browser.open('https://dev.example.com/inner/path/get_cookie.html')
>>> browser.cookies['boo']
'boo'
>>> del browser.cookies['boo']
>>> browser.cookies['boo']
'hoo'
Creating a cookie that will be immediately masked within the current url is not allowed.
>>> browser.cookies.getinfo('tweedle')['path']
'/inner/path'
>>> browser.cookies.create('tweedle', 'dum', path='/inner')
...
Traceback (most recent call last):
...
ValueError: cannot set a cookie that will be hidden by another cookie for
this url (https://dev.example.com/inner/path/get_cookie.html)
>>> browser.cookies['tweedle']
'dee'
Masking by Domain¶
All of the same behavior is also true for domains. The only difference is a theoretical one: while the behavior of masking cookies via paths is defined by the relevant IRCs, it is not defined for domains. Here, we simply follow a “best match” policy.
We initialize by setting some cookies for example.org.
>>> browser.open('https://dev.example.org/get_cookie.html')
>>> browser.cookies.keys() # a different domain
[]
>>> browser.cookies.create('tweedle', 'dee')
>>> browser.cookies.create('boo', 'yah', domain='example.org',
... secure=True)
Before we look at the examples, note that the default behavior of the cookies is to be liberal in the matching of domains.
>>> browser.cookies.strict_domain_policy
False
According to the RFCs, a domain of ‘example.com’ can only be set implicitly
from the server, and implies an exact match, so example.com URLs will get
the cookie, but not *.example.com
(i.e., dev.example.com
). Real
browsers vary in their behavior in this regard. The cookies collection, by
default, has a looser interpretation of this, such that domains are always
interpreted as effectively beginning with a “.”, so dev.example.com
will
include a cookie from the example.com
domain filter as if it were a
.example.com
filter.
Here’s an example. If we go to dev.example.org
, we should only see the
“tweedle” cookie if we are using strict rules. But right now we are using
loose rules, so ‘boo’ is around too.
>>> browser.open('https://dev.example.org/get_cookie.html')
>>> sorted(browser.cookies)
['boo', 'tweedle']
>>> print(browser.contents)
boo: yah
tweedle: dee
If we set strict_domain_policy
to True, then only tweedle is included.
>>> browser.cookies.strict_domain_policy = True
>>> sorted(browser.cookies)
['tweedle']
>>> browser.open('https://dev.example.org/get_cookie.html')
>>> print(browser.contents)
tweedle: dee
If we set the “boo” domain to .example.org
(as it would be set under
the more recent Cookie RFC if a server sent the value) then maybe we get
the “boo” value again.
>>> browser.cookies.forURL('https://example.org').change(
... 'boo', domain=".example.org")
Traceback (most recent call last):
...
ValueError: policy does not allow this cookie
Whoa! Why couldn’t we do that?
Well, the strict_domain_policy affects what cookies we can set also. With strict rules, “.example.org” can only be set by “.example.org” domains, *not example.org itself.
OK, we’ll create a new cookie then.
>>> browser.cookies.forURL('https://snoo.example.org').create(
... 'snoo', 'kums', domain=".example.org")
>>> sorted(browser.cookies)
['snoo', 'tweedle']
>>> browser.open('https://dev.example.org/get_cookie.html')
>>> print(browser.contents)
snoo: kums
tweedle: dee
Let’s set things back to the way they were.
>>> del browser.cookies['snoo']
>>> browser.cookies.strict_domain_policy = False
>>> browser.open('https://dev.example.org/get_cookie.html')
>>> sorted(browser.cookies)
['boo', 'tweedle']
>>> print(browser.contents)
boo: yah
tweedle: dee
Now back to the the examples of masking by domain. First, unless you pass an explicit domain, you will be modifying the existing cookie.
>>> browser.cookies.getinfo('boo')['domain']
'example.org'
>>> browser.cookies['boo'] = 'hoo'
>>> browser.cookies.getinfo('boo')['domain']
'example.org'
>>> browser.cookies.getinfo('boo')['secure']
True
Now we mask the cookie, using the domain.
>>> browser.cookies.create('boo', 'boo', domain='dev.example.org')
>>> browser.cookies['boo']
'boo'
>>> browser.cookies.getinfo('boo')['domain']
'dev.example.org'
>>> browser.cookies.getinfo('boo')['secure']
False
>>> browser.cookies['boo']
'boo'
>>> sorted(browser.cookies.keys())
['boo', 'tweedle']
To identify the additional cookies, you can change the URL…
>>> extra_cookies = browser.cookies.forURL(
... 'https://example.org/get_cookie.html')
>>> extra_cookies['boo']
'hoo'
>>> extra_cookies.getinfo('boo')['domain']
'example.org'
>>> extra_cookies.getinfo('boo')['secure']
True
…or use iterinfo
and pass in a name.
>>> pprint.pprint(list(browser.cookies.iterinfo('boo')))
[{'comment': None,
'commenturl': None,
'domain': 'dev.example.org',
'expires': None,
'name': 'boo',
'path': '/',
'port': None,
'secure': False,
'value': 'boo'},
{'comment': None,
'commenturl': None,
'domain': 'example.org',
'expires': None,
'name': 'boo',
'path': '/',
'port': None,
'secure': True,
'value': 'hoo'}]
An odd situation in this case is that deleting a cookie can sometimes reveal another one.
>>> browser.open('https://dev.example.org/get_cookie.html')
>>> browser.cookies['boo']
'boo'
>>> del browser.cookies['boo']
>>> browser.cookies['boo']
'hoo'
Setting a cookie with a foreign domain from the current URL is not allowed (use forURL to get around this).
>>> browser.cookies.create('tweedle', 'dum', domain='locahost.local')
Traceback (most recent call last):
...
ValueError: current url must match given domain
>>> browser.cookies['tweedle']
'dee'
Setting a cookie that will be immediately masked within the current url is also not allowed.
>>> browser.cookies.getinfo('tweedle')['domain']
'dev.example.org'
>>> browser.cookies.create('tweedle', 'dum', domain='.example.org')
...
Traceback (most recent call last):
...
ValueError: cannot set a cookie that will be hidden by another cookie for
this url (https://dev.example.org/get_cookie.html)
>>> browser.cookies['tweedle']
'dee'
change
¶
So far all of our examples in this section have centered on create
.
change
allows making changes to existing cookies. Changing expiration
is a good example.
>>> browser.open("http://localhost/@@/testbrowser/cookies.html")
>>> browser.cookies['foo'] = 'bar'
>>> browser.cookies.change('foo', expires=datetime.datetime(2031, 1, 1))
>>> browser.cookies.getinfo('foo')['expires']
datetime.datetime(2031, 1, 1, 0, 0, tzinfo=<UTC>)
That’s the main story. Now here are some edge cases.
>>> browser.cookies.change(
... 'foo',
... expires=zope.testbrowser.cookies.expiration_string(
... datetime.datetime(2030, 1, 1)))
>>> browser.cookies.getinfo('foo')['expires']
datetime.datetime(2030, 1, 1, 0, 0, tzinfo=<UTC>)
>>> browser.cookies.forURL(
... 'http://localhost/@@/testbrowser/cookies.html').change(
... 'foo',
... expires=zope.testbrowser.cookies.expiration_string(
... datetime.datetime(2029, 1, 1)))
>>> browser.cookies.getinfo('foo')['expires']
datetime.datetime(2029, 1, 1, 0, 0, tzinfo=<UTC>)
>>> browser.cookies['foo']
'bar'
>>> browser.cookies.change('foo', expires=datetime.datetime(1999, 1, 1))
>>> len(browser.cookies)
4
While we are at it, it is worth noting that trying to create a cookie that has already expired raises an error.
>>> browser.cookies.create('foo', 'bar',
... expires=datetime.datetime(1999, 1, 1))
Traceback (most recent call last):
...
AlreadyExpiredError: May not create a cookie that is immediately expired
Clearing cookies¶
clear, clearAll, clearAllSession allow various clears of the cookies.
The clear
method clears all of the cookies for the current page.
>>> browser.open('http://localhost/@@/testbrowser/cookies.html')
>>> len(browser.cookies)
4
>>> browser.cookies.clear()
>>> len(browser.cookies)
0
The clearAllSession
method clears all session cookies (for all domains
and paths, not just the current URL), as if the browser had been restarted.
>>> browser.cookies.clearAllSession()
>>> len(browser.cookies)
0
The clearAll
removes all cookies for the browser.
>>> browser.cookies.clearAll()
>>> len(browser.cookies)
0
Note that explicitly setting a Cookie header is an error if the cookies
mapping has any values; and adding a new cookie to the cookies
mapping
is an error if the Cookie header is already set. This is to prevent hard-to-
diagnose intermittent errors when one header or the other wins.
>>> browser.cookies['boo'] = 'yah'
>>> browser.addHeader('Cookie', 'gee=gaw')
Traceback (most recent call last):
...
ValueError: cookies are already set in `cookies` attribute
>>> browser.cookies.clearAll()
>>> browser.addHeader('Cookie', 'gee=gaw')
>>> browser.cookies['fee'] = 'fi'
Traceback (most recent call last):
...
ValueError: cookies are already set in `Cookie` header