Pages

Friday, February 16, 2018

How DNS work

Step 1. The browser sends a query to the local DNS server in your network.

Step 2. If the local network has no internal DNS server or the DNS server has no information about the site you want to visit, a query is sent to the recursive DNS server of your Internet service provider (ISP). If the recursive server has information about the IP address in its cache, your browser receives that information. No additional queries are performed.

Step 3. However, if the recursive server does not know the IP address, the recursive server sends the query to one of the 13 sets of root name servers located worldwide. A root server knows the DNS information about a top-level domain (TLD).

Step 4. The DNS server of a TLD sends information about the second-level domain and its authoritative name server. An authoritative name server knows all the addressing information for a particular domain.

Step 5. The authoritative name server responds to a query by returning the Address Record (A record) to your ISP. The ISP’s recursive server stores the record on its cache for a specific amount of time and sends the IP address to your browser.

Friday, December 15, 2017

Python OOP Concepts

Variable concept:

a = 10
a is a variable that refers to object 10.
same is for function definition
def test
a test is a variable which refers to object function test
variable is reference or label to object not placeholder
All attributes are called by "." operator
Anything defined inside class is class attributes
Anything defined inside function is variable
only variable has scope
every class should be instantiated

while True is a substitution of do-while in python

# only variable have scope but attribute doesn’t have
## variable defined under class are class attributes. variable defined inside and outside functions are local and global variables
## list and dictionary don’t store data. They refer to data
## variables are not object in python. variables are reference to object
## all definition happens in run time
## any object ref count is 0 is freed from memory i.e. garbage collection
## you can’t have two scope under a function at a same time
## import this (for python zen)
## python is compiled interpreter language not line by line interpreter language
## i ++ is not supported in python
## to check object is callable or not
>>> v = 10
>>> callable(v)
False
>>> def foo(): print("hello world")
...
>>> callable(foo)
True
## class is a instance of type

## tuple is ", separator operator

>>> t = 1,2,3,4,5,6
>>>
>>> t
(1, 2, 3, 4, 5, 6)

or

>>> t = (1,2,3,4,5)
>>> t
(1, 2, 3, 4, 5)

## we can’t have one element tuple
>>> t = (1)  ## this is basically integer
>>> t
1
>>> type(t)

>>>

If you want to have one element tuple use
>>> t = (1,)
>>> t
(1,)
>>> type(t)


dict:
>>> d = { 'one': 1, 'two': 2, 'three': 3}
>>> d
{'one': 1, 'two': 2, 'three': 3}
>>>
>>>
>>> d = dict(one = 1, two = 2, three = 3)
>>> d
{'one': 1, 'two': 2, 'three': 3}
>>>
>>> type(d)

>>>
>>> x = dict(four = 4, five =5, six = 6)
>>> x
{'four': 4, 'five': 5, 'six': 6}
>>>
>>> d = dict(one = 1, two = 2, three =3, **x)
>>> d
{'one': 1, 'two': 2, 'three': 3, 'four': 4, 'five': 5, 'six': 6}
>>>
>>>
>>> 'four' in x
True
>>> 'three' in x
False
>>>
>>>
>>> for k in d: print(k)
...
one
two
three
four
five
six
>>>
>>>
>>> for k, v in d.items():print(k, v)
...
one 1
two 2
three 3
four 4
five 5
six 6
>>>
>>>
>>> d['three']
3
>>>
>>>
>>> x['three']
Traceback (most recent call last):
  File "", line 1, in
KeyError: 'three'
>>>
>>>
>>> x.get('three')
>>> x
{'four': 4, 'five': 5, 'six': 6}
>>>
>>>
>>> d.get('three')
3
>>>
>>>
>>> x.get('three', 'not found')
'not found'
>>>
>>> x
{'four': 4, 'five': 5, 'six': 6}
>>>
>>>
>>> del x['four']
>>> x
{'five': 5, 'six': 6}
>>>
>>>
>>> x.pop('five')
5
>>>
>>>
>>>
>>>
>>> x
{'six': 6}
>>>

## iter function is a function that returns iterator object
## Note: yield turns function into generator. Each time yield is run it returns
## the value and next time the function is called, execution starts right after yield and this turns function into generator
## and what it generates is iterator object which can be used like any iterator object in python.

# define a list
my_list = [4, 7, 0, 3]

# get an iterator using iter()
my_iter = iter(my_list)

## iterate through it using next()

#prints 4
print(next(my_iter))

#prints 7
print(next(my_iter))

## next(obj) is same as obj.__next__()

#prints 0
print(my_iter.__next__())

#prints 3
print(my_iter.__next__())

## This will raise error, no items left
next(my_iter)

two ways to create dict in python:
>>> d = {'one': 1, 'two': 2, 'three': 3, 'four': 4, 'five': 5}
>>> for k in d:
...     print(k, d[k])
...
one 1
two 2
three 3
four 4
five 5
>>>
>>>
>>> for k in sorted(d.keys()):
...     print(k,d[k])
...
five 5
four 4
one 1
three 3
two 2
>>>
>>> d = dict(
...     one =1, two = 2, three = 3, four =4, five = 'five'
... )
>>>
>>> for k in sorted(d.keys()):
...     print(k,d[k])
...
five five
four 4
one 1
three 3
two 2
>>>
>>>
>>> d['seven'] = 7
>>> for k in sorted(d.keys()):
...     print(k, d[k])
...
five five
four 4
one 1
seven 7
three 3
two 2

>>> d = { 'one': 1, 'two': 2, 'three': 3}
>>> d
{'one': 1, 'two': 2, 'three': 3}
>>>
>>>
>>> d = dict(one = 1, two = 2, three = 3)
>>> d
{'one': 1, 'two': 2, 'three': 3}
>>>
>>> type(d)

>>>
>>> x = dict(four = 4, five =5, six = 6)
>>> x
{'four': 4, 'five': 5, 'six': 6}
>>>
>>>
>>> d = dict(one = 1, two = 2, three =3, **x)
>>> d
{'one': 1, 'two': 2, 'three': 3, 'four': 4, 'five': 5, 'six': 6}
>>>
>>>
>>> 'four' in x
True
>>> 'three' in x
False
>>>
>>>
>>> for k in d: print(k)
...
one
two
three
four
five
six
>>>
>>>
>>> for k, v in d.items():print(k, v)
...
one 1
two 2
three 3
four 4
five 5
six 6
>>>
>>>
>>> d['three']
3
>>>
>>>
>>> x['three']
Traceback (most recent call last):
  File "", line 1, in
KeyError: 'three'
>>>
>>>
>>> x.get('three')
>>> x
{'four': 4, 'five': 5, 'six': 6}
>>>
>>>
>>> d.get('three')
3
>>>
>>>
>>> x.get('three', 'not found')
'not found'
>>>
>>> x
{'four': 4, 'five': 5, 'six': 6}
>>>
>>>
>>> del x['four']
>>> x
{'five': 5, 'six': 6}
>>>
>>>
>>> x.pop('five')
5
>>>
>>>
>>>
>>>
>>> x
{'six': 6}
>>>

In python NOT has first precedence, then AND then OR

# define a list
my_list = [4, 7, 0, 3]

# get an iterator using iter()
my_iter = iter(my_list)

## iterate through it using next()

#prints 4
print(next(my_iter))

#prints 7
print(next(my_iter))

## next(obj) is same as obj.__next__()

#prints 0
print(my_iter.__next__())

#prints 3
print(my_iter.__next__())

## This will raise error, no items left
next(my_iter)

### for is used for deterministic loop
[email protected]:/users/nawlekha/Desktop/pyATS/Python_OOP$ python3 -v findall_exercise.py
## for verbose print

## Instance of base class is type and inheritance of base class is object

Iterator in Python is simply an object that can be iterated upon. An object which will return data, one element at a time.
Technically speaking, Python iterator object must implement two special methods, __iter__() and __next__(), collectively called the iterator protocol.
An object is called iterable if we can get an iterator from it. Most of the built-in containers in Python like: list, tuple, string etc. are iterable.
The iter() function (which in turn calls the __iter__() method) returns an iterator from them.

The match() function only checks if the RE matches at the beginning of the string while search() will scan forward through the string for a match. It’s important to keep this distinction in mind. Remember, match() will only report a successful match which will start at 0; if the match wouldn’t start at zero, match() will not report it.
>>>
>>> print(re.match('super', 'superstition').span())
(0, 5)
>>> print(re.match('super', 'insuperable'))
None
On the other hand, search() will scan forward through the string, reporting the first match it finds.
>>>
>>> print(re.search('super', 'superstition').span())
(0, 5)
>>> print(re.search('super', 'insuperable').span())
(2, 7)

>>> a = dict(one=1, two=2, three=3)
>>> b = {'one': 1, 'two': 2, 'three': 3}
>>> c = dict(zip(['one', 'two', 'three'], [1, 2, 3]))
>>> d = dict([('two', 2), ('one', 1), ('three', 3)])
>>> e = dict({'three': 3, 'one': 1, 'two': 2})
>>> a == b == c == d == e
True

To print python builtins:
>>> dir(__builtins__)

>>> list(range(5,3))
[]
>>>

>>> 'ab c\n\nde fg\rkl\r\n'.splitlines()
['ab c', '', 'de fg', 'kl']
>>>
>>>
>>>
>>> 'ab c\n\nde fg\rkl\r\n'.split()
['ab', 'c', 'de', 'fg', 'kl']
>>>

>>> "One line\n".splitlines()
['One line']
>>>
>>>
>>> 'Two lines\n'.split('\n')
['Two lines', '']
>>>

## default inheritance of all class is from object
that’s in python3

python2
class Employee(object): pass

python3
class Employee(): pass

## class attributes are commonly shared among all instances and automatically inherited to their instances. Instances can overwrite them

##if we make a callable method as class attribute, they are automatically wired up as an instance method
## for instance method to work, the first argument should be always referenced to the instance

## all functions defined under __init__ are local variables
if we want them globally we can create instance attribute
all instance attributes are visible globally

## this will not work
>>> class Car:
...     def __init__(self):
...             print("car object created...")
...     def drive():
...             print("driving a car....")
...
>>> c = Car()
car object created...
>>> c.drive()
Traceback (most recent call last):
  File "", line 1, in
TypeError: drive() takes 0 positional arguments but 1 was given
>>>

c.drive() is equivalent to Car.drive(c)

but Car.drive() will work in python3.
>>> Car.drive()
driving a car....

To make it work in both pyhon2 and python3, we can use staticmethod

calling drive either from object or from class should work. But this is not at all good practice to follow
>>> class Car:
...     def __init__(self):
...             print("car object created...")
...     @staticmethod
...     def drive():
...             print("driving a car....")
...
>>> c = Car()
car object created...
>>> c.drive()
driving a car....
>>> Car.drive()
driving a car....
>>>

Other decorator is @classmethod

## without classmethod decorator

>>> class Car:
...     def __init__(self):
...             print("car object created...")
...     @staticmethod
...     def drive():
...             print("driving a car....")
...     def sell(c):
...             print("c =", c)
...

>>> c = Car()
car object created...
>>> c.drive()
driving a car....
>>> c.sell()
c = <__main__ .car="" 0x108c30d30="" at="" object="">
## sell takes objet instance

Now let’s use @classmethod

>>> class Car:
...     def __init__(self):
...             print("car object created...")
...     @staticmethod
...     def drive():
...             print("driving a car....")
...     @classmethod
...     def sell(c):
...             print("c =", c)
...
>>> c = Car()
car object created...
>>> c.drive()
driving a car....
>>> Car.drive()
driving a car....
>>> c.sell()
c =
## it will always pass class as an argument to this particular instance
c.sell() or Car.sell() means the same
>>> Car.sell()
c =

staticmethod will pass nothing it will call function as it is. a static method cannot access neither class or instance attributes.

## trying to find value in dictionary with unknown key will raise a error

>>> a = {"name": "john", "role": "admin"}
>>> a
{'name': 'john', 'role': 'admin'}
>>>
>>>
>>> a["name"]
'john'
>>>
>>> a["city"]
Traceback (most recent call last):
  File "", line 1, in
KeyError: 'city'

## if we don’t want to raise an error. we can use get function. it will return default value None for unknown key

>>> a.get("name")
'john'
>>>
>>> a.get("city")
>>>
>>> v = a.get("city")
>>> print(v)
None
>>>

## instead of default value None we can also decide what value to print

>>> v = a.get("city", "not found")
>>> v
'not found'
>>>

but

>>> a.get("role", "not found")
'admin'
>>>

run script in debugger mode + python prompt
Python_OOP$ python3 -m pdb scope_1.py
> /Users/nawlekha/Desktop/pyATS/Python_OOP/scope_1.py(10)()
-> """
(Pdb) help

Documented commands (type help ):
========================================
EOF    c          d        h         list      q        rv       undisplay
a      cl         debug    help      ll        quit     s        unt
alias  clear      disable  ignore    longlist  r        source   until
args   commands   display  interact  n         restart  step     up
b      condition  down     j         next      return   tbreak   w
break  cont       enable   jump      p         retval   u        whatis
bt     continue   exit     l         pp        run      unalias  where

Miscellaneous help topics:
==========================
exec  pdb

(Pdb) n
>Python_OOP/scope_1.py(113)()
-> """
(Pdb) s

l == list a program
n = next will step over a function
s = will step into a function
globals = to list all global variables currently defined


function prototype:
==================

>>> def greet():
...     print("Hello world")
...
>>> def greet(name):
...     print("Welcome", name)
...
>>> greet("John")
Welcome John
>>> greet()
Traceback (most recent call last):
  File "", line 1, in
TypeError: greet() missing 1 required positional argument: 'name'
>>>

Rem: the last definition overwrites the previous definition
similar to
a = 10
a = “hello world”

Both greet variable points to different functions and last overwrites the previous

So how to do polymorphism
=============
a function with an ability to work differently based on different arguments.

>>> def greet(name=None):
...     if name is None:
...             print("Hello world")
...     else:
...             print("Welcome", name)
...
>>> greet("John")
Welcome John
>>> greet()
Hello world

or

>>> def greet(name="Guest"):
...     print("Welcome", name)
...
>>> greet("John")
Welcome John
>>> greet()
Welcome Guest


## we should always pass non-default argument followed by default argument
>>> def greet(name="Guest", city):
...     print("Hello", name, "Welcome to", city)
...
  File "", line 1
SyntaxError: non-default argument follows default argument
>>>


>>> def greet(name, city="bangalore"):
...     print("Hello", name, "Welcome to", city)
...
>>> greet("John", "Mumbai")
Hello John Welcome to Mumbai
>>>
>>> greet("John")
Hello John Welcome to bangalore
>>>

>>> def greet(name="Guest", city="bangalore"):
...     print("hello", name, "Welcome to", city)
...
>>> greet()
hello Guest Welcome to bangalore
>>>

## print takes care of adding space after “,” and add new line after last argument

>>> def greet(name="Guest", city="bangalore"):
...     print("hello", name, "Welcome to", city)
...
>>> greet("John")
hello John Welcome to bangalore

>>> greet("Mumbai")
hello Mumbai Welcome to bangalore
## python doesn’t know Mumbai is name of city. It just replace first argument with Mumbai

>>> greet(city="Mumbai")
hello Guest Welcome to Mumbai
## this is what we call as passing key-word argument

## solution is explicitly mention keyword arguments
>>> greet(city="Pune", name="Smith")
hello Smith Welcome to Pune

## all python programs can be run as the script or loaded as a module.
## if you run a program as a script, name of a file is __main__ and everything in the file will execute including the main function.
## if you load program as a module, codes outside of “if” statement will only execute i.e. program main function will not run.

[email protected]:~/Desktop$ vi test.py
def greet(): print("Hello world")
name = "John"

print("__name__=", __name__)

if __name__ == "__main__":
        greet()
        print("name=", name)

[email protected]:~/Desktop$ python3 test.py
__name__= __main__
Hello world
name= John

[email protected]:~/Desktop$
[email protected]:~/Desktop$ python3
Python 3.6.2 |Anaconda, Inc.| (default, Sep 21 2017, 18:29:43)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import test
__name__= test
>>> test.greet()
Hello world
>>> test.name
'John'
>>>

## string is a sequence and immutable. Other sequences in python are list and tuples.
All sequence should be indicible

## to have mutable string we can use bytearray

>>> a = "Hello world"
>>> a
'Hello world'
>>> a[0]
'H'
>>> a[0] = "A"
Traceback (most recent call last):
  File "", line 1, in
TypeError: 'str' object does not support item assignment
>>>
>>> a = bytearray(b"Hello world")
>>> a
bytearray(b'Hello world')
>>>
>>> a[0]
72 ## this is ascii value of H
>>> a[0] = 65
>>> a
bytearray(b'Aello world') ## this mutates existing string
>>>

## Two kinds of immutable strings. Unicode and byte string
Strings by default represent sequence of unicode characters
>>> a = "hello world"
>>> type(a)

>>>
In unicode, each character can be of different bytes may be 1, 2 or 3 byte size based on encoding.This supports different language characters and it is UTF compliance.This is useful to read textual data.

but to read binary data i.e. packet, binary buffer, element must be byte.Each element must be a byte.
to create byte string

>>> b = b"Hello world"
>>> b
b'Hello world'
>>> type(b)

>>> print(b)
b'Hello world'
## this is a sequence of bytes

## byte array can only be created from byte string.

## Variable argument function. Function that can take one or more arguments
## arbitrary argument list a.k.a varargs

>>> def greet(*args):
...     pass

or

>>> def greet(*users):
...     pass

## args is not a keyword. Non of the function parameters are keywords. self is also not a reserved keyword. args and kwargs are just a convention.

* followed by name indicates collect all arguments in the form of tuple

>>> def greet(*users):
...     print("users=", users)
...
>>> greet("John", "smith", "Sam", "Joe")
users= ('John', 'smith', 'Sam', 'Joe') ## this is tuple

>>> def greet(name, city="Pune", *visited):
...     print(name)
...     print(city)
...     print(visited)
...
>>>
>>> greet()
Traceback (most recent call last):
  File "", line 1, in
TypeError: greet() missing 1 required positional argument: 'name'
>>> greet("John")
John
Pune
()
>>>
>>> greet("John", "bengaluru", "Kolkatta", "Chennai", "Mumbai")
John
bengaluru
('Kolkatta', 'Chennai', 'Mumbai')

## so rule is no-defualt, followed by default followed by varargs. This is for python2
but here limitation is we can’t print default value.if we use default argument with varargs, the default argument is no longer a default

but in python3 we can have non-default followed by varargs and default


>>> def greet(name, *visited, city="Pune"):
...     print(name)
...     print(city)
...     print(visited)
...
>>> greet("John")
John
Pune
()
>>> greet("John", "Mumbai", "Kolkatta", 'Hyderabad') ## default argument always remain default

John
Pune
('Mumbai', 'Kolkatta', 'Hyderabad')
>>>
>>>
>>> greet("John", "Mumbai", "Kolkatta", 'Hyderabad', city= "Bengaluru") ## now overwrite default value
John
Bengaluru
('Mumbai', 'Kolkatta', 'Hyderabad')

## so in python3.4 and above non-default should be the first argument always and varargs and default arguments are interchangeable

## How to pass variable number of arguments

>>> def store_record(name, role, dept, city):
...     print("Storing name = {}, role= {}, dept = {}, city= {}".format(name, role, dept,city))
...
>>> rec = "John", "Support", "IT", "Bengaluru"
>>>
>>> store_record(*rec)
Storing name = John, role= Support, dept = IT, city= Bengaluru

## numbers of arguments in function and numbers of values in tuple used while calling should be same

>>> rec = "John", "Support", "IT"
>>>
>>> store_record(*rec)
Traceback (most recent call last):
  File "", line 1, in
TypeError: store_record() missing 1 required positional argument: 'city'
>>>

>>> rec = "John", "Support", "IT", "Mumbai", "Bengaluru" ## this can be either list or tuple
>>>
>>> store_record(*rec)
Traceback (most recent call last):
  File "", line 1, in
TypeError: store_record() takes 4 positional arguments but 5 were given


## tuple is extension of list

## Advantage of format
>>> a = "Hello %s, How are you %s, What will you do today %s"
>>> a
'Hello %s, How are you %s, What will you do today %s'
>>> print(a % ("John", "John", "John"))
Hello John, How are you John, What will you do today John
>>>
>>>
>>> b = "Hello {0}, How are you {0}, What will do you today {0}"
>>> b.format("John")
'Hello John, How are you John, What will do you today John'
>>>

>>> a = "Decimal: {0:d}, Hex: {0:x}, Oct: {0:o}, Bin: {0:b}"
>>> a
'Decimal: {0:d}, Hex: {0:x}, Oct: {0:o}, Bin: {0:b}'
>>>
>>> a.format(34)
'Decimal: 34, Hex: 22, Oct: 42, Bin: 100010'
>>>

{0:d} == %d
{0:x} == %x

>>> def store_data(*n): ## collects arguments as tuple
...     print(n)
...
>>> store_data()
() ## empty tuple
>>>

arbitrary keyword arguments
>>> def store_data(**n): ## collects arguments as dictionary
...     print(n)
...
>>> store_data()
{} ## empty dictionary

>>> store_data(name="John", a=100, test="Helloworld", info=(10,20,30))
{'name': 'John', 'a': 100, 'test': 'Helloworld', 'info': (10, 20, 30)}

>>>
>>> def store_data(a, b=20, *c, **d):
...     print("a=", a)
...     print("b=", b)
...     print("c=", c)
...     print("d=", d)
...
>>> store_data(10)
a= 10
b= 20
c= ()
d= {}
>>>

>>> store_data(10,20,30,40,50,60,70)
a= 10
b= 20
c= (30, 40, 50, 60, 70)
d= {}
>>>

## all positional arguments got collected in tuple

>>> store_data(10,20,30,40,50,60,70, x=10, name= "Sam", role= "Admin")
a= 10
b= 20
c= (30, 40, 50, 60, 70)
d= {'x': 10, 'name': 'Sam', 'role': 'Admin'}

## all keyword arguments got collected in dictionary

>>> def store_record(name, role, dept, city):
...     print("Storing name = {}, role = {}, dept = {}, city = {}".format(name, role, dept, city))
...
>>> rec = {"name": "John", "dept": "IT", "city": "Bengaluru", "role": "Support"}

>>> store_record(**rec)
Storing name = John, role = Support, dept = IT, city = Bengaluru

## when using ** whatever variables you are passing there should be a mapping


## for default value
>>> def store_record(name, role, dept, city="Pune"):
...     print("Storing name = {}, role = {}, dept = {}, city = {}".format(name, role, dept, city))
...
>>> rec = {"name": "John", "dept": "IT", "role": "Support"}
>>>
>>> store_record(**rec)
Storing name = John, role = Support, dept = IT, city = Pune
>>>


## let’ say we have hundreds and function and all are returning some string
Once function is called we want to convert all return strings into upper case

## one way is we can change return string under all functions. other is while calling we have to do call.upper() but this will be tedious since we need to do it across all the function

Solution is:

define one more function which can do it.
This is called a decorator. It takes once function and return another function

def greet():
    return "Hello world"

def welcome():
    return "Welcome to Python"

def convert_to_upper(fn):
    def wrapper():
        return fn().upper()
    return wrapper


print("greet =", greet) ## this will print a reference since we are not calling function
print(greet())  ## calling original greet function
greet = convert_to_upper(greet)
print("greet =", greet) ## reference to convert_to_upper function
print(greet()) ## returns wrapper


runfile('/Users/nawlekha/Desktop/pyATS/decorator-working.py', wdir='/Users/nawlekha/Desktop/pyATS')
greet =
Hello world
greet = .wrapper at 0x1811e5f0d0>
HELLO WORLD

## This is called as implementing cross-cutting function

print("greet =", greet)
print(greet())
print(welcome)
print(welcome())
greet = convert_to_upper(greet)
welcome = convert_to_upper(welcome)
print("greet =", greet)
print(greet())
print(welcome())

runfile('/Users/nawlekha/Desktop/pyATS/decorator-working.py', wdir='/Users/nawlekha/Desktop/pyATS')
greet =
Hello world

Welcome to Python
greet = .wrapper at 0x1811e5f378>
HELLO WORLD
WELCOME TO PYTHON


## this syntax is simplified by python

def convert_to_upper(fn):
    def wrapper():
        return fn().upper()
    return wrapper

@convert_to_upper
def greet():
    return "Hello world"

@convert_to_upper ## decorator function, @ is called annotation. annotation can be placed before fun or class
def welcome():
    return "Welcome to Python"

print("greet =", greet)
print("welcome =", welcome)

print(greet())
print(welcome())

runfile('/Users/nawlekha/Desktop/pyATS/decorator-working.py', wdir='/Users/nawlekha/Desktop/pyATS')
greet = .wrapper at 0x1811e5fae8> 
welcome = .wrapper at 0x1811e5f378>
HELLO WORLD
WELCOME TO PYTHON

Here is how it works.
decorator will take greet() function as an argument and whatever it will return is assigned to greet variable.

## callable objects: functions, classes, and method underclasses are callable objects

## understand the flow
def convert_to_upper(fn):
    print("convert_to_upper called: fn =", fn)
    def wrapper():
        return fn().upper()
    return wrapper

@convert_to_upper ##while defining itself greet is called
def greet():
    return "Hello world"

print(greet())
print(greet())

## Here decorator is called only once. The number of time annotation is used, decorator is called.

runfile('/Users/nawlekha/Desktop/pyATS/decorator-working.py', wdir='/Users/nawlekha/Desktop/pyATS')
convert_to_upper called: fn =
HELLO WORLD
HELLO WORLD


def to_upper(fn):
    def wrapper(n):
        return fn(n).upper()
    return wrapper

@to_upper
def greet(name):
    return "Hello " + name

print(greet("John"))

runfile('/Users/nawlekha/Desktop/pyATS/decorator-working.py', wdir='/Users/nawlekha/Desktop/pyATS')
HELLO JOHN

Generalization
==============

def to_upper(fn):
    def wrapper(*args, **kwargs):
        return fn(*args, **kwargs).upper()
    return wrapper

@to_upper
def greet(name):
    return "Hello " + name


@to_upper
def welcome(name,city):
    return "Hello {}. Welcome to {}".format(name, city)

print(greet("John"))

print(welcome("Smith", "Bengaluru"))


HELLO JOHN
HELLO SMITH. WELCOME TO BENGALURU


Partials:
==========

def store_record(name, role, dept, city):
    print("Storing name = {}, role = {}, dept = {}, city = {}".format(name, role, dept, city))


store_record("John", "admin", "IT", "Bengaluru")
store_record("John", "admin", "IT", "Pune")
store_record("John", "admin", "IT", "Mumbai")
store_record("John", "admin", "IT", "Kolkatta")

Storing name = John, role = admin, dept = IT, city = Bengaluru
Storing name = John, role = admin, dept = IT, city = Pune
Storing name = John, role = admin, dept = IT, city = Mumbai
Storing name = John, role = admin, dept = IT, city = Kolkatta

## passing first three common arguments multiple times is cumbersome here

Solution is hardcode first three arguments

def partial_deco(fn):
    def wrapper(city):
        return fn("John", "admin", "IT", city)
    return wrapper

@partial_deco
def store_record(name, role, dept, city):
    print("Storing name = {}, role = {}, dept = {}, city = {}".format(name, role, dept, city))


#store_record("John", "admin", "IT", "Bengaluru")
#store_record("John", "admin", "IT", "Pune")
#store_record("John", "admin", "IT", "Mumbai")
#store_record("John", "admin", "IT", "Kolkatta")

store_record("Bengaluru")

Storing name = John, role = admin, dept = IT, city = Bengaluru

## other variation

def partial_deco(fn):
    def wrapper(**kwargs):
        defaults = {"name": "John", "role": "admin", "dept": "IT"}
        if "city" not in kwargs:
            raise TypeError("city is required...")
        defaults.update(kwargs)
        return fn(**defaults)
    return wrapper

@partial_deco
def store_record(name, role, dept, city):
    print("Storing name = {}, role = {}, dept = {}, city = {}".format(name, role, dept, city))


#store_record("John", "admin", "IT", "Bengaluru")
#store_record("John", "admin", "IT", "Pune")
#store_record("John", "admin", "IT", "Mumbai")
#store_record("John", "admin", "IT", "Kolkatta")

store_record(city="Bengaluru")
store_record(name = "Smith", city="Bengaluru")
store_record(name = "Smith")

Storing name = John, role = admin, dept = IT, city = Bengaluru
Storing name = Smith, role = admin, dept = IT, city = Bengaluru
TypeError: city is required...

## if you want to initialize instance attribute the moment object is created, put them under __init__ method.

NameError: looking for global variable which is not defined
AttributeError: trying to access attribute which is not defined in an object

test(100)

NameError: name 'test' is not defined

os.getcwd()

NameError: name 'os' is not defined

sys.owner

AttributeError: module 'sys' has no attribute 'owner'

=========
>>> def greet():
...     a = 100
...     val = 500
...     name = "Smith"
...     print(name, "says 'Hello'")
...
>>>
>>> greet

>>>
>>>
>>> a = greet
>>>
>>>
>>> a

>>>
>>> del greet ## deleting greet will not delete greet function. There is still reference to greet function i.e. a
>>>
>>> greet
Traceback (most recent call last):
  File "", line 1, in
NameError: name 'greet' is not defined
>>>
>>>
>>> a

>>>
>>>

>>> a()
Smith says 'Hello'
>>>
>>>
>>> a

>>>
>>> dir(a)
['__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']
>>>

>>> a.__qualname__ ## name used during function definition
'greet'

we can change it to some different name

>>> a.__qualname__ = "new_function"
>>>
>>> a

>>>


>>> a.__code__
", line 1>

## to find local variables defined in this function

>>> a.__code__.co_varnames
('a', 'val', 'name')
>>>

## to find byte code in binary format
>>> a.__code__.co_code
b'd\x01}\x00d\x02}\x01d\x03}\x02t\x00|\x02d\x04\x83\x02\x01\x00d\x00S\x00'

facade pattern
==============

class Car:
    def __init__(self, name,price):
        self.name, self.price = name, price
 
    def drive(self):
        print("Driving", self.name)


c = Car("honda", 10000)

c
Out[13]: <__main__ .car="" 0x108bbf3c8="" at="">

dir(c)
Out[14]:
['__class__',
 '__delattr__',
 '__dict__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__',
 'drive',
 'name',
 'price']

c.name
Out[15]: 'honda'

c.price
Out[16]: 10000

c.drive()
Driving honda

c.color
Traceback (most recent call last):

  File "", line 1, in
    c.color

AttributeError: 'Car' object has no attribute 'color'

## we can handle this attribute error

class Car:
    def __init__(self, name,price):
        self.name, self.price = name, price
 
    def drive(self):
        print("Driving", self.name)
 
    def __getattr__(self, attr):
        print("Get-attr called!")
        return 100


Now if any attributes is not found and if we have defined __getattr__ function it will call this function

c = Car("Honda", 1000000000)

c.name
Out[31]: 'Honda'

c.price
Out[32]: 1000000000

c.color
Get-attr called!
Out[33]: 100

c.owner
Get-attr called!
Out[34]: 100

## all attributes are stored as dictionary in instance

Instead of that we want to store it somewhere else, in outside dict

attrs = {}

class Car:
    def __init__(self, name, price):
        attrs[id(self)] = {}
        attrs[id(self)]['name'] = name
        attrs[id(self)]['price'] = price
 
    def drive(self):
        print("Driving", self.name)
 
    def __getattr__(self, attr):
        return attrs[id(self)][attr]

    def __setattr__(self, attr, value):
        attrs[id(self)][attr]] = value


attrs
Out[36]: {}

c = Car("Honda", 100000)

attrs
Out[38]: {4441491888: {'name': 'Honda', 'price': 100000}}

c2 = Car("Maruti", 50000)

attrs
Out[42]:
{4441491888: {'name': 'Honda', 'price': 100000},
 4441567752: {'name': 'Maruti', 'price': 50000}}

c.name ## getting value but attribute is not there in dir(c). It's getting from outside dict
Out[45]: 'Honda'

dir(c)
Out[46]:
['__class__',
 '__delattr__',
 '__dict__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattr__',
 '__getattribute__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__',
 'drive']

=========
## private methods. Not accessible from outside world

class Car:
    def __sell(self):
        print("Car sold...")
    def drive(self):
        self.__sell()

c = Car()

c.drive()
Car sold...

c.__sell()
Traceback (most recent call last):

  File "", line 1, in
    c.__sell()

AttributeError: 'Car' object has no attribute '__sell'

Note : if you define anything with __word then it is not accessible to outside world.

There is a different way to access it.

c._Car__sell
Out[59]: >

c._Car__sell()
Car sold...

Unicode vs byte array in Python
========================
Suppose you create a string using Python 3 in the usual way:
stringobject = 'ant'
stringobject would be a unicode string.
A unicode string is made up of unicode characters. In string object above, the unicode characters are the individual letters, e.g. a, n, t
Each unicode character is assigned a code point, which can be expressed as a sequence of hex digits (a hex digit can take on 16 values, ranging from 0-9 and A-F). For instance, the letter 'a' is equivalent to '\u0091', and 'ant' is equivalent to '\u0061\u006E\u0074'.
So you will find that if you type in,
stringobject = '\u0061\u006E\u0074'
stringobject
You will also get the output 'ant'.
Now, unicode is converted to bytes, in a process known as encoding. The reverse process of converting bytes to unicode is known as decoding.
How is this done? Since each hex digit can take on 16 different values, it can be reflected in a 4-bit binary sequence (e.g. the hex digit 0 can be expressed in binary as 0000, the hex digit 1 can be expressed as 0001 and so forth). If a unicode character has a code point consisting of four hex digits, it would need a 16-bit binary sequence to encode it.
Different encoding systems specify different rules for converting unicode to bits. Most importantly, encodings differ in the number of bits they use to express each unicode character.
For instance, the ASCII encoding system uses only 8 bits (1 byte) per character. Thus it can only encode unicode characters with code points up to two hex digits long (i.e. 256 different unicode characters). The UTF-8 encoding system uses 8 to 32 bits (1 to 4 bytes) per character, so it can encode unicode characters with code points up to 8 hex digits long, i.e. everything.
Running the following code:
byteobject = stringobject.encode('utf-8')
byteobject, type(byteobject)
converts a unicode string into a byte string using the utf-8 encoding system, and returns b'ant', bytes'.
Note that if you used 'ASCII' as the encoding system, you wouldn't run into any problems since all code points in 'ant' can be expressed with 1 byte. But if you had a unicode string containing characters with code points longer than two hex digits, you would get a UnicodeEncodeError.
Similarly,
stringobject = byteobject.decode('utf-8')
stringobject, type(stringobject)
gives you 'ant', str.

The only thing that a computer can store is bytes.
To store anything in a computer, you must first encode it, i.e. convert it to bytes. For example:
If you want to store music, you must first encode it using MP3, WAV, etc.
If you want to store a picture, you must first encode it using PNG, JPEG, etc.
If you want to store text, you must first encode it using ASCII, UTF-8, etc.
MP3, WAV, PNG, JPEG, ASCII and UTF-8 are examples of encodings. An encoding is a format to represent audio, images, text, etc in bytes.
In Python, a byte string is just that: a sequence of bytes. It isn't human-readable. Under the hood, everything must be converted to a byte string before it can be stored in a computer.
On the other hand, a character string, often just called a "string", is a sequence of characters. It is human-readable. A character string can't be directly stored in a computer, it has to be encoded first (converted into a byte string). There are multiple encodings through which a character string can be converted into a byte string, such as ASCII and UTF-8.
'I am a string'.encode('ASCII')
The above Python code will encode the string 'I am a string' using the encoding ASCII. The result of the above code will be a byte string. If you print it, Python will represent it as b'I am a string'. Remember, however, that byte strings aren't human-readable, it's just that Python decodes them from ASCII when you print them. In Python, a byte string is represented by a b, followed by the byte string's ASCII representation.
A byte string can be decoded back into a character string if you know the encoding that was used to encode it.
b'I am a string'.decode('ASCII')
The above code will return the original string 'I am a string'.
Encoding and decoding are inverse operations. Everything must be encoded before it can be written to disk, and it must be decoded before it can be read by a human.

If you are running file directly then file name is __main__
if you are running file by importing first file then it will print first file name

Generator doesn’t hold entire result in memory.
Generator is better with performance











Sunday, October 22, 2017

Python Beyond Basics

arithmetic operations
c = a + b   # c = a.__add__(b)
c = a - b   # c = a.__sub__(b)
c = a * b   # c = a.__mul__(b)
c = a / b   # c = a.__div__(b)
c = a % b   # c = a.__mod__(b)
c = a // b  # c = a.__floordiv__(b)
c = a ** b  # c = a.__pow__(b)

c = -a      # c = a.__neg__()

c = a | b   # c = a.__or__(b)
c = a & b   # c = a.__and__(b)
c = a ^ b   # c = a.__xor__(b)
c = ~a     

c = a << 1  # c = a.__lshift__(1)
c = a >> 1  # c = a.__rshift__(1)
===

assignments
Simple statements
---------------------
1. Simple assignment
   a = 10
   b = 20

2. Tuple packing
   a = 10, 20, 30

3. Tuple unpacking
   b, c, d = a

4. Parallel assignment
   a, b = 10, 20

5. Assignment chainloading
   a = b = c = 10
 
6. Augmented assignments
   a += 10
----------------------------
del variable_name

print variable, expression
====

boolean expression
 ==
 != <>
 <
 >
 <=
 >=

 is
 is not

 and
 or
 not

 a == b
a < b
a > b
a <= b
a >= b

a != b
a <> b   # only in python2

a is b
a is not b

element in collection
"with" in "this is a test string with some words"
10 in (33, 44, 10, 44, 33, 44)

not in

not
and
or

conditional expression

a == b
a != b, a <> b
a > b
a < b
a >= b
a <= b

a is b
a is not b

33 in (11, 22, 33, 44, 55, 66)

"test" not in "this is a dummy string"

and
or
not

if not a == b
if a != b
 ======

 boolean context

  0, 0.0, "", (), [], {}, False, None -> False
=====

1. Boolean comparison operators
-------------------------------
    ==
    != <>
    <
    >
    <=
    >=

    is
    is not

    in
    not in

    Examples
    --------
        >>> a = 10
        >>> b = 10
        >>> a == b
        True
        >>>
     
        >>> c = 20
        >>> a > c
        False
        >>> a < c
        True
        >>> a != c
        True

2. Logical evaluation operators
-------------------------------
    and
    or
    not

    Examples
    --------
    >>> a = 10
    >>> b = 20
    >>> c = 30
    >>> b > a and b < c
    True
    >>> a == 10 or b == 100
    True

3. Special boolean operators
----------------------------
    is
    is not

        Example
        -------
        >>> a = "Hello world"
        >>> b = "Hello world"
        >>> c = a
        >>> a is b
        False
        >>> a is c
        True
        >>> a is not b
        True

    in
    not in

        Example
        -------
        >>> a = 33, 44, 55, 32, 12, 56, 77
        >>> 44 in a
        True
        >>> 100 not in a
        True
        >>> 67 in a
        False

4. Boolean context
------------------
false_values = False, None, 0, 0.0, "", (), [], {}
====

built-in function types
1. Type constructors (type conversion functions)
2. Generalized functions (work on broader range of object types)
3. Introspection functions
====

 If you have a quote in the middle of the string? Python needs help to recognize quotes as part of the English language and not as part of the Python language.
>>> “I can’t do that”
“I can’t do that”
 >>> “He said \“no\” to me”
“He said “no” to me”


Conditional statements:
=======================
    ##############################################################
    if [boolean-expression]:
     
    elif [another-boolean-expression]:
           

    else:
       
       
>>> num = 21
>>> if num == 20:
... print 'the number is 20'
... elif num > 20:
... print 'the number is greater then 20'
... else:
... print 'the number is less then 20'
...
the number is greater then 20


    ###############################################################
    if [boolean-expression]:

>>> if num == 10:
...     print("The number is 10")
... 

The number is 10

    ###############################################################
    if [boolean-expression] else

>>> num = 20
>>> if num == 20:
... print 'the number is 20'
... else:
... print 'the number is not 20'
...
the number is 20


Loops:
======
    ###############################################################
    while [boolean-expression]:
       
       
        if [some-condition]: break
        if [some-other-condition]: continue
       
        ....
       
    else:
       
>>> num = 1
>>> while num <= 5:
...     print(num)
...     num += 1
... 
1
2
3
4

5

>>> i = 1
>>> while i < 6:
...     print(i)
...     if i == 3:
...             break
...     i += 1
... 
1
2
3


>>> i = 0
>>> while i < 6:
...     i += 1
...     if i == 3:
...             continue
...     print(i)
... 
1
2
4
5
6

    ###############################################################
    for element in iterable:
       
       
        if [some-condition]: break
        if [some-other-condition]: continue
       
        ...
       
    else:
       

>>> colors = ('red','blue','green')
>>> 
>>> colors
('red', 'blue', 'green')
>>> 
>>> for favorite in colors:
...     print("I love", favorite)
... 
I love red
I love blue

I love green

>>> fruits = ["apple", "banana", "cherry"]
>>> for x in fruits:
...     if x == "banana":
...             break
...     print(x)
... 

apple


>>> fruits = ["apple", "banana", "cherry"]
>>> for x in fruits:
...     if x == "banana":
...             continue
...     print(x)
... 
apple
cherry

Exception handling:
===================
    try:
       
       
    except SomeError as error_object:
       
    except SomeOtherError as error_object:
       
    except Exception as error_object:
       
    else:
       
    finally:
       

Context management:
===================
    with as object:
       
     
Function definition:
====================
    def function_name():
       
>>> def my_function():
...   print("Hello from a function")
... 
>>> my_function()
Hello from a function

>>> 

Class definition:
=================
    class ClassName:
       
       ....

Create a class named Person, use the __init__() function to assign values for name and age:

class Person:
  def __init__(self, name, age):
    self.name = name
    self.age = age

p1 = Person("John", 36)

print(p1.name)
print(p1.age)


=====

Data types

NoneType -> None

bool -> True, False

Numeric
   int, float, long, complex
   fractions.Decimal, fractions.Fraction

Iterable Objects
   Collection
      Sequences            -> Ordered collection of elements
         str (String)      -> "hello", 'hello', """hello""", r'hello'
         unicode           -> u'hello world'
         tuple             -> (10, "hello", 3.4)
-----------------------------------------------------------------------------------
         bytearray         -> bytearray('hello world')
         list              -> [10, "hello", 3.4]
         collections.deque -> deque([22, 33, 44, 55])

      Set                  -> Unordered collection of unique hashable objects
         set               -> {33, 44, 54, 12, 56}

      MappingType          -> Unordered collection of key:value pair
         dict              -> {
                               "name" : "Smith",
                               "city" : "New York",
                               "host" : "44.33.21.45",
                               "port" : 8080
                              }
         collections.OrderedDict

   xrange object
   File objects
   Generators
   Views
====

definitions
1. Assignment statement
-----------------------
a = 100
name = "john"

2. function definition
----------------------
def greet(): print "Hello world"

3. module definition / module loading
-------------------------------------
import sys
from sys import version

4. class definition
-------------------
class Person: pass

To remove variable:
   del variablename
=====

equality
==      a == 10, b == "john"
!=
<
>
<=
>=

in      "john" in names,
not in  10 not in numbers

is      -> check whether two variables refer to same object
is not
========

exception

Parse Errors (errors while parsing the python program):
   - SyntaxError
   - IndentationError

Runtime Errors
--------------
    NameError -> invalid variable name or access to undefined variable
    TypeError -> invalid type detected or object type is incompatible
    ValueError -> incompatible value detected.
    IndexError -> invalid index in a sequence
    KeyError   -> invalid key in a map
    AttributeError -> invalid attribute on an object

    IOError -> invalid file or socket I/O operation
    OSError -> error while executing underlying OS syscall
======

for loop
a = "Hello"
for c in a: print c,

H
e
l
l
o

it = iter(a)

try:
   while True:
      c = it.next()
      print c,
except StopIteration:
   pass
 
Traceback (most recent call last):
  File "", line 3, in
AttributeError: 'str_iterator' object has no attribute 'next' 
=====

Identifiers
In Python, identifiers can be:
  a. variable name
  b. function name
  c. class name
  d. module name
  e. constants

Naming conventions
------------------
1. Variables
   count
   name
 
   word_count
   program_name

2. Functions
   is_prime()
   generate_primes()

   gen_values()

3. Classes
   EmployeeInfo

4. Modules
   sys
   os
   flask

5. Constants
   ---------
   MAX = 100
   MAX_VALUE = 500
====

Immutable types
NoneType (None)
bool (True / False)
numbers (int, float, long, complex)
sequences -> str, unicode, tuple
set -> frozenset
=====

Introspection function
type(variable)
id(variable)
dir(variable)
help(variable)
help(variable.attribute)
====

naming conventions
All variable names should start with lower-case and contain alphabets and _
    max_count
    user_name

functions must start with a verb
    run_command
    start_process
    kill_process
    wait_for_thread

Module names must also start with lower-case and contain alphabets and _
    import time
    import sys

boolean functions start with is_
    is_prime()
    is_lower()

All classes follow CamelCase convention and start with Capital letter
    CustomerOrder
    InvoiceInfo

Constants are all capital letters
    MAX_COUNT
    PI_VALUE
====

1. All variables are references to Objects
2. All identifiers are variables
   a. Function name
   b. Class name
   c. Module name
   d. Other variable names

3. All assignments are "assign by references"
4. Every object is uniquely identified by the object-id.
=====

numeric operater
+
-
*
/
%

**   power
//   floordiv
====

Object types
Immutable objects
Hashable objects
Iterable objects
     Collection
        -> Sequence : An ordered collection of items
             str (string)
             unicode
             bytearray
             tuple           
=====

references
Variables
Attributes of an object
Index of a sequence
Key of a mapping
:
====

sequence operator
+    Concatenation
*    Repetition
%    String format operator
====

sequence function
sum()
max(), min(), all(), any()

sorted(), reversed()

enumerate(), zip(), itertools.izip()

iter()

len()
====

sequences
Sequences -> str, unicode, tuple, list, bytearray, buffer
Sequence operations
-------------------
  +  concatenation
     a = 10, 20, 30
     b = 40, 50, 60
     c = a + b
     print c
     10, 20, 30, 40, 50, 60

  * repetition
     a = 10, 20
     b = a * 5
     print b
     10, 20, 10, 20, 10, 20, 10, 20 ...

  [] indexing
     a[0]
     a[-1]
     a[2:3]
     a[::2]

Sequence methods
----------------
   a.index(element)
   a.count(element)

General sequence functions
--------------------------
len(s)
min(s)
max(s)
all(s)
any(s)

sum(s)

zip(s1, s2)
enumerate(s)

reversed(s)
sorted(s)
=====

statements
1. print tuple          # Prints each element of a tuple as a string
   print "Hello world", "another string", 10, 5.6

2. Assignments
   a = 10               # simple assignment
   a = 10, 20           # tuple packing
   b, c = a             # tuple unpacking
   a, b = 10, 20        # parallel assignment
   a += 5               # augmented assignment
   a = b = c = 10       # assignment chain-loading

3. del variable_or_ref  # Remove a variable or a reference 
   del a                # Remove the variable 'a' and
                        # decrement the reference count
                        # of the object referred by 'a'

4. import module        # Load a module and in the module namespace

   import sys           # Load a module called 'sys'
   sys.version          # Access members of 'sys' module using 'sys' namespace

   import sys as system # Load a module called 'sys' under 'system' namespace
   system.version       # Access member called 'version' from
                        # 'system' namespace

5. from module import member  # Load a module and make members of that module
                              # accessible in the current namespace

   from sys import getsizeof  # getsizeof() function can now be accessed
                              # directly without namespace resolution.


   from sys import getsizeof as sizeof    # import a member named 'getsizeof'
                                          # from module named 'sys' as a
                                            new name 'sizeof'

6. pass                 # Represent a do-nothing statement

7. raise SomeError      # Raise a runtime exception of type SomeError

8. assert expression    # Assert expression to be true
-------------------------------------------------------------------------------
Other simple statements
-----------------------
9.  return value        # Return a value from a function
10. break               # Break out of a while/for loop
11. continue            # Skip to next iteration of while/for loop
12. yield value         # Yield a value to the generator
13. global variable     # Declare a variable in a function to be global
===

special methods
+   __add__
-   __sub__
*   __mul__
/   __div__
**  __pow__
//  __floordiv__
%   __mod__

<<  __lshift__
>>  __rshift__
|   __or__
&   __and__
^   __xor__
~   __neg__
+= __iadd__    # a += 1 --> a.__iadd__(1)
-= __isub__
....
a == b
a.__cmp__(b) == 0

a > b
a.__cmp__(b) > 0

a < b
a.__cmp__(b) < 0
====

strings
a = "He said - I'll be back"
a = 'He said "I will be back" and we waited'
a = "He said - \"I'll be back\" and we waited"
a = 'He said - "I\'ll be back" and we waited'

a = """He said - "I'll be back" and we waited"""

install_path = r'C:\backup\remote\network\temp\assorted'
unicode_data = u'Hello world'
---------------------------------------------------------
String operators
----------------
  %  - String format operator
  +  - String concatenation (from sequence)
  *  - String repetition (from sequence)
  [] - String index operator (from sequence)
 ====

 oop test

 from __future__ import print_function

class Car:
    def __init__(self, name):
        self.name = name

    @staticmethod
    def sell():
        print("Sold this car...")

    @classmethod
    def count(c):
        print("c =", c)

    def drive(self):
        print("Driving", self.name)

c = Car("Honda")
c.drive()
c.sell()
Car.sell()
c.count()
Car.count()

Driving Honda
Sold this car...
Sold this car...
c =
c =
=====

Performance check

"""
A simple python program to test CPU performance by
generating a series of prime numbers
"""
from __future__ import print_function
import sys
if sys.version_info[0] == 2:
    from __builtin__ import xrange as range

NUM_PRIMES = 100000

def is_prime(number):
    "Returns True if 'number' is prime"
    limit = int(number ** 0.5) + 1
    for i in range(2, limit):
        if number % i == 0:
            return False
    return True

def gen_prime(num):
    "Generates first 'num' series of prime numbers"
    i = 2
    while num:
        if is_prime(i):
            print(i)
            num -= 1
        i += 1

from time import time

print("Generating %d prime numbers..." % NUM_PRIMES)
start = time()
gen_prime(NUM_PRIMES)
duration = time() - start
print("gen_prime(%d) took %f seconds" % (NUM_PRIMES, duration))
           
gen_prime(100000) took 12.242300 seconds
=====

list vs tuple vs set 
from timeit import timeit
from sys import argv

sample_size = int(argv[1]) if len(argv) > 1 else 1000

num_iters = 100000
datatypes = "tuple", "list", "set"

setup_code = """
from random import randint
a = {datatype}(range({limit}))
"""
stmt_code = "randint(0, {limit}) in a"

for dt in datatypes:
    duration = timeit(
                  setup=setup_code.format(datatype=dt, limit=sample_size),
                  stmt=stmt_code.format(limit=sample_size),
                  number=num_iters)

    print("Searching for a random element in {} sized {} took {} seconds".format(
                                           sample_size, dt, duration))
                                         
##output                                         
Searching for a random element in 1000 sized tuple took 0.7152920619992074 seconds
Searching for a random element in 1000 sized list took 0.737770362989977 seconds
Searching for a random element in 1000 sized set took 0.14120669901603833 seconds                        ====

Thread performance
from __future__ import print_function
from threading import Thread

from performance import profile_time, print_log
import sys

if sys.version_info[0] == 2:
    range = xrange

def worker(w):
    y = 0
    print("Starting worker: {}\n".format(w), end="")
    for i in range(1000):
        for j in range(10000):
            y += i*j
    print("Worker {} complete: {}\n".format(w, y), end="")

@profile_time
def start_workers():
    pool = { }
    for i in range(16):
        pool[i] = Thread(target=worker, args=(i,))
        pool[i].start()
    print("Created 16 workers...\n", end="")

    for i in range(16):
        pool[i].join()
    print("All workers complete...\n", end="")

start_workers()

print_log()
===

read struct
from __future__ import print_function
import struct

silly_data_format = 'ich64s'
with open("silly_data.dat", "rb") as source:
    data = struct.unpack(silly_data_format, source.read())
    print(data)

#output
(42, b'a', 67, b"hello world from the 'C'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00")
=====

read line 

import readline

while True:
    line = input("Enter a line: ")
    print(line) 
===

local vs global
a = 10
def foo():
    a = 20
    print("In foo: a =", a)
    print(globals())
    print("-" * 30)
    print(locals())
    globals()['a'] = 40

foo()
print("In main: a =", a)

##output
In foo: a = 20
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external .sourcefileloader="" 0x10821f780="" at="" object="">, '__spec__': None, '__annotations__': {}, '__builtins__': , '__file__': 's1.py', '__cached__': None, 'a': 10, 'foo': }
------------------------------
{'a': 20}
In main: a = 40
====

##scope oo
color = "white"

def start():
    print("Started program...")

class Car:
    color = "red"

    def __init__(self):
        self.color = "green"

    def start(self):
        print("Started car...")

    def drive(self):
        start()
        print("Driving a", color, "car")
        print("Car.color =", Car.color)
        print("self.color =", self.color)

c = Car()
c.drive()

##output
Started program...
Driving a white car
Car.color = red
self.color = green 
=====

a = 10

def foo():
    print("In foo: a =", a)
    a = 40

foo()
print("In main: a =", a)

##output
Traceback (most recent call last):
  File "scope.py", line 7, in
    foo()
  File "scope.py", line 4, in foo
    print("In foo: a =", a)
UnboundLocalError: local variable 'a' referenced before assignment
====

from __future__ import print_function

a = 10

def foo():
    global a
    print("In foo: a =", a)
    a = 20

foo()
print("In main: a =", a)

##output 
In foo: a = 10
In main: a = 20
===

a = [10, 20, 30]

def foo():
    print("In foo: a =", a)
    a[0] = 100

foo()
print("In main: a =", a)

##output
In foo: a = [10, 20, 30]
In main: a = [100, 20, 30]
====

color = "white"

def start():
    print("Starting program...")

class Car:
    color = "blue"

    def __init__(s):
        s.color = "green"

    def start(self):
        print("Starting car...")

    def drive(self):
        self.start()
        print("Driving a", color, "car")
        print("Car.color =", Car.color)
        print("self.color =", self.color)

c = Car()
c.drive()

print(color)
print(Car.color)
print(c.color)

##output
Starting car...
Driving a white car
Car.color = blue
self.color = green
white
blue
green
=====

a = [10, 20, 30]
b = [40, 50, 60]

from itertools import chain

for i in chain(a, b):
    print(i)

##output
10
20
30
40
50
60
====

#print table

num = int(input("Enter a number: "))
for i in 1, 2, 3, 4, 5, 6, 7, 8, 9, 10:
    print(num, "x", i, "=", num*i)
 
##output
Enter a number: 5
5 x 1 = 5
5 x 2 = 10
5 x 3 = 15
5 x 4 = 20
5 x 5 = 25
5 x 6 = 30
5 x 7 = 35
5 x 8 = 40
5 x 9 = 45
5 x 10 = 50
====

print("Hello world")
name = "John"
print("Name is {}".format(name))
print("Square of 2 is", square(2))

def square(x): return x*x
===

from foo import square
print("square of 9 is", square(9))
====

color = "white"

class Car:
    color = "red"

    def __init__(self):
        self.color = "green"

    def drive(self):
        print("Driving a", color, "car")
        print("Car.color =", Car.color)
        print("self.color =", self.color)

c = Car()
c.drive()

Driving a white car
Car.color red
self.color =  green
=====

chaining
a = [10, 20, 30, 40]
b = [50, 60, 70, 80]
from itertools import chain

for i in chain(a, b): print i
====

cyclic reference

class Car:
    def __init__(self):
        print("Car object created...")

    def __del__(self):
        print("Car object destroyed...")

class Person:
    def __init__(self):
        print("Person object created...")

    def __del__(self):
        print("Person object destroyed...")

p = Person()
c = Car()
c.owner = p
p.owns = c

print("Created car and person...")
del c
del p
print("Variables c and p deleted...")
import gc
print(gc.collect())

##output
Person object created...
Car object created...
Created car and person...
Variables c and p deleted...
Person object destroyed...
Car object destroyed...
4
=========

delete 

class Person:

    def __init__(self):
        print("User object created...")

    def __del__(self):
        print("User object destroyed...")

class Car:

    def __init__(self):
        print("Car object created...")

    def __del__(self):
        print("Car object destroyed...")

p = Person()
c = Car()

p.owns = c
c.owner = p

print("Car and Person object created...")

#output
User object created...
Car object created...
Car and Person object created...
User object destroyed...
Car object destroyed...
=======

dict:
user_info = dict(name="John",
                 role="Admin",
                 dept="IT",
                 city="Bengaluru")

for key, value in user_info.items():
    print("{} -> {}".format(key, value))

#or
user_info = {"name": "nawraj","role": "Engineer","dept": "Eng"}

for key, value in user_info.items():
print("{} -> {}".format(key, value))

=====

duck typing

a = [10, 20, 30]
b = (10, 20, 30)

def c():
    print("Hello world")

class d:
    def __init__(self):
        print("Created an object...")

objs = a, b, c, d

print(objs)

def run(tasks):
    for t in tasks:
        if callable(t): t()

run(objs)

#output
([10, 20, 30], (10, 20, 30), , )
Hello world
Created an object...
====

findall 

"""
Preliminary exercise
====================
Implement the findall() function below that must return a list
of indices all occurrence of a substring found in a string
(both passed as arguments to the function).

Example usage:
--------------
   >>> quote = '''
   ... When I see a bird
   ... that walks like a duck
   ... and swims like a duck
   ... and quacks like a duck,
   ... I call that bird a duck
   ... '''

   >>> findall(quote, "duck")
   [37, 59, 82, 107]

"""

def findall(main_string: str, sub_string: str) -> list:
    """
    Returns a list of indices of each occurrence of
    sub_string in main_string

    Example usage:
    --------------
        >>> poem = '''
        ... A fly and flea flew into a flue,
        ... said the fly to the flea 'what shall we do ?'
        ... 'let us fly' said the flea
        ... and said the fly 'let us flee'
        ... and so they flew through a flaw in to the flue.
        ... '''

        >>> findall(poem, 'fly')
        [3, 43, 88, 120]

    """
    indices = []
    i = 0
    while True:
        i = main_string.find(sub_string, i)
        if i == -1: break
        indices.append(i)
        i += len(sub_string)
    return indices

if __name__ == '__main__':
    import doctest
    doctest.testmod()
    # Running this program using 'python3 findall.py' should
    # ideally pass all tests in the doctest - on implementation
    # of findall() function
======

name = "smith"

class User:
    """
    A simple User class

    eoiwru iowe uriowue rio wuerio
    sdlj fskldj fklsdj flksdjfkls

    """
    def __init__(self, name):
        """
            Create a new user object with 'name' passed
            as argument.
        """
        self.name = name

    def greet(self): print(self.name, "says Hello!")

    def welcome(): print("Welcome to Python...")

def square(x):
    "Returns square of argument x."
    return x*x

if __name__ == '__main__':
    print("Main program running...")
    print("a =", a)
    print("name =", name)
    print("square(2) =", square(2))
    u = User("John")
    u.greet()

##otuput 
Main program running...
a = 100
name = smith
square(2) = 4
John says Hello!    
=========

for loop implementation

a = [10, 20, 30, 40, 50]

#for i in a: print i,

iterator = iter(a)
if hasattr(a, '__getitem__') and hasattr(a, '__len__'):
    length = len(a)
    index = 0
    while index < length:
        i = a[index]
        index += 1

if hasattr(a, '__len__'):  # For fixed-length iterable objects
    length = len(a)        # (a.k.a collections/containers)
    while length:
        i = next(iterator)
        print (i),  # 'for' loop body
        length -= 1

else:                      # For generators/views/non-collection iterables
    try:
        while True:
            i = next(iterator)
            print (i),  # 'for' loop body
    except StopIteration:
        pass

#output
10
20
30
40
50
========

class Car:
    def __init__(self):
        print("Created a new car object...")

    def __del__(self):
        print("Car object is being destroyed!")

c1 = Car()
c2 = c1
c3 = c2
print(c1, c2, c3)

print("Changing c2...")
c2 = 100

print("Deleting c1...")
del c1

print("Changing c3...")
c3 = "Hello world"

print("End of program...")

##output
Created a new car object...
<__main__ .car="" 0x109a13780="" at="" object=""> <__main__ .car="" 0x109a13780="" at="" object=""> <__main__ .car="" 0x109a13780="" at="" object="">
Changing c2...
Deleting c1...
Changing c3...
Car object is being destroyed!
End of program...
=====

a = "10"

def foo():
    from builtins import int
    la = a

    for i in range(1000000):
        print("Value of a is", int(la))

foo()
========

def greet():
    print("Hello world")

def greet(user):
    print("Hello", user)

greet("John")
greet()

##output
Hello John
Traceback (most recent call last):
  File "greet_bad.py", line 8, in
    greet()
TypeError: greet() missing 1 required positional argument: 'user'
====

def greet(user=None):
    if user is None:
        print("Hello world")
    else:
        print("Hello", user)

greet("John")
greet()

Hello John
Hello world

========
use of sep

print("Hello world", "Another string", sep="")

## sep is the separator used between multiple values when printing. The default is a space (sep=' ')
===

use fo end

print("Hello world", end="\t")

# Python’s print() function comes with a parameter called ‘end’. By default, the value of this parameter is ‘\n’, i.e. the new line character. You can end a print statement with any character/string using this parameter
===

Infinite loop
from itertools import count
for i in count(10, 2):
    print(i)

# count(start=0, step=1) --> count object

from itertools import count

i = 0
while True:
    print(i)
    i += 1

for i in count():
    print(i)
====

module test

def foo():
    from time import ctime
    print("Time now is", ctime())

def bar():
    from time import ctime
    print("Time now in bar is", ctime())

foo()
bar()

Time now is Fri Oct 20 10:12:17 2017
Time now in bar is Fri Oct 20 10:12:17 2017

## ctime : Convert a time in seconds since the Epoch to a string in local time.
    This is equivalent to asctime(localtime(seconds)). When the time tuple is
    not present, current time as returned by localtime() is used.