## Sunday, October 22, 2017

### Python Beyond Basics

arithmetic operations
c = a + b   # c = a.__add__(b)
c = a - b   # c = a.__sub__(b)
c = a * b   # c = a.__mul__(b)
c = a / b   # c = a.__div__(b)
c = a % b   # c = a.__mod__(b)
c = a // b  # c = a.__floordiv__(b)
c = a ** b  # c = a.__pow__(b)

c = -a      # c = a.__neg__()

c = a | b   # c = a.__or__(b)
c = a & b   # c = a.__and__(b)
c = a ^ b   # c = a.__xor__(b)
c = ~a

c = a << 1  # c = a.__lshift__(1)
c = a >> 1  # c = a.__rshift__(1)
===

assignments
Simple statements
---------------------
1. Simple assignment
a = 10
b = 20

2. Tuple packing
a = 10, 20, 30

3. Tuple unpacking
b, c, d = a

4. Parallel assignment
a, b = 10, 20

a = b = c = 10

6. Augmented assignments
a += 10
----------------------------
del variable_name

print variable, expression
====

boolean expression
==
!= <>
<
>
<=
>=

is
is not

and
or
not

a == b
a < b
a > b
a <= b
a >= b

a != b
a <> b   # only in python2

a is b
a is not b

element in collection
"with" in "this is a test string with some words"
10 in (33, 44, 10, 44, 33, 44)

not in

not
and
or

conditional expression

a == b
a != b, a <> b
a > b
a < b
a >= b
a <= b

a is b
a is not b

33 in (11, 22, 33, 44, 55, 66)

"test" not in "this is a dummy string"

and
or
not

if not a == b
if a != b
======

boolean context

0, 0.0, "", (), [], {}, False, None -> False
=====

1. Boolean comparison operators
-------------------------------
==
!= <>
<
>
<=
>=

is
is not

in
not in

Examples
--------
>>> a = 10
>>> b = 10
>>> a == b
True
>>>

>>> c = 20
>>> a > c
False
>>> a < c
True
>>> a != c
True

2. Logical evaluation operators
-------------------------------
and
or
not

Examples
--------
>>> a = 10
>>> b = 20
>>> c = 30
>>> b > a and b < c
True
>>> a == 10 or b == 100
True

3. Special boolean operators
----------------------------
is
is not

Example
-------
>>> a = "Hello world"
>>> b = "Hello world"
>>> c = a
>>> a is b
False
>>> a is c
True
>>> a is not b
True

in
not in

Example
-------
>>> a = 33, 44, 55, 32, 12, 56, 77
>>> 44 in a
True
>>> 100 not in a
True
>>> 67 in a
False

4. Boolean context
------------------
false_values = False, None, 0, 0.0, "", (), [], {}
====

built-in function types
1. Type constructors (type conversion functions)
2. Generalized functions (work on broader range of object types)
3. Introspection functions
====

If you have a quote in the middle of the string? Python needs help to recognize quotes as part of the English language and not as part of the Python language.
>>> “I can’t do that”
“I can’t do that”
>>> “He said \“no\” to me”
“He said “no” to me”

Conditional statements:
=======================
##############################################################
if [boolean-expression]:

elif [another-boolean-expression]:

else:

>>> num = 21
>>> if num == 20:
... print 'the number is 20'
... elif num > 20:
... print 'the number is greater then 20'
... else:
... print 'the number is less then 20'
...
the number is greater then 20

###############################################################
if [boolean-expression]:

>>> if num == 10:
...     print("The number is 10")
...

The number is 10

###############################################################
if [boolean-expression] else

>>> num = 20
>>> if num == 20:
... print 'the number is 20'
... else:
... print 'the number is not 20'
...
the number is 20

Loops:
======
###############################################################
while [boolean-expression]:

if [some-condition]: break
if [some-other-condition]: continue

....

else:

>>> num = 1
>>> while num <= 5:
...     print(num)
...     num += 1
...
1
2
3
4

5

>>> i = 1
>>> while i < 6:
...     print(i)
...     if i == 3:
...             break
...     i += 1
...
1
2
3

>>> i = 0
>>> while i < 6:
...     i += 1
...     if i == 3:
...             continue
...     print(i)
...
1
2
4
5
6

###############################################################
for element in iterable:

if [some-condition]: break
if [some-other-condition]: continue

...

else:

>>> colors = ('red','blue','green')
>>>
>>> colors
('red', 'blue', 'green')
>>>
>>> for favorite in colors:
...     print("I love", favorite)
...
I love red
I love blue

I love green

>>> fruits = ["apple", "banana", "cherry"]
>>> for x in fruits:
...     if x == "banana":
...             break
...     print(x)
...

apple

>>> fruits = ["apple", "banana", "cherry"]
>>> for x in fruits:
...     if x == "banana":
...             continue
...     print(x)
...
apple
cherry

Exception handling:
===================
try:

except SomeError as error_object:

except SomeOtherError as error_object:

except Exception as error_object:

else:

finally:

Context management:
===================
with as object:

Function definition:
====================
def function_name():

>>> def my_function():
...   print("Hello from a function")
...
>>> my_function()
Hello from a function

>>>

Class definition:
=================
class ClassName:

....

Create a class named Person, use the __init__() function to assign values for name and age:

class Person:
def __init__(self, name, age):
self.name = name
self.age = age

p1 = Person("John", 36)

print(p1.name)
print(p1.age)

=====

Data types

NoneType -> None

bool -> True, False

Numeric
int, float, long, complex
fractions.Decimal, fractions.Fraction

Iterable Objects
Collection
Sequences            -> Ordered collection of elements
str (String)      -> "hello", 'hello', """hello""", r'hello'
unicode           -> u'hello world'
tuple             -> (10, "hello", 3.4)
-----------------------------------------------------------------------------------
bytearray         -> bytearray('hello world')
list              -> [10, "hello", 3.4]
collections.deque -> deque([22, 33, 44, 55])

Set                  -> Unordered collection of unique hashable objects
set               -> {33, 44, 54, 12, 56}

MappingType          -> Unordered collection of key:value pair
dict              -> {
"name" : "Smith",
"city" : "New York",
"host" : "44.33.21.45",
"port" : 8080
}
collections.OrderedDict

xrange object
File objects
Generators
Views
====

definitions
1. Assignment statement
-----------------------
a = 100
name = "john"

2. function definition
----------------------
def greet(): print "Hello world"

-------------------------------------
import sys
from sys import version

4. class definition
-------------------
class Person: pass

To remove variable:
del variablename
=====

equality
==      a == 10, b == "john"
!=
<
>
<=
>=

in      "john" in names,
not in  10 not in numbers

is      -> check whether two variables refer to same object
is not
========

exception

Parse Errors (errors while parsing the python program):
- SyntaxError
- IndentationError

Runtime Errors
--------------
TypeError -> invalid type detected or object type is incompatible
ValueError -> incompatible value detected.
IndexError -> invalid index in a sequence
KeyError   -> invalid key in a map
AttributeError -> invalid attribute on an object

IOError -> invalid file or socket I/O operation
OSError -> error while executing underlying OS syscall
======

for loop
a = "Hello"
for c in a: print c,

H
e
l
l
o

it = iter(a)

try:
while True:
c = it.next()
print c,
except StopIteration:
pass

Traceback (most recent call last):
File "", line 3, in
AttributeError: 'str_iterator' object has no attribute 'next'
=====

Identifiers
In Python, identifiers can be:
a. variable name
b. function name
c. class name
d. module name
e. constants

Naming conventions
------------------
1. Variables
count
name

word_count
program_name

2. Functions
is_prime()
generate_primes()

gen_values()

3. Classes
EmployeeInfo

4. Modules
sys
os

5. Constants
---------
MAX = 100
MAX_VALUE = 500
====

Immutable types
NoneType (None)
bool (True / False)
numbers (int, float, long, complex)
sequences -> str, unicode, tuple
set -> frozenset
=====

Introspection function
type(variable)
id(variable)
dir(variable)
help(variable)
help(variable.attribute)
====

naming conventions
All variable names should start with lower-case and contain alphabets and _
max_count
user_name

run_command
start_process
kill_process

Module names must also start with lower-case and contain alphabets and _
import time
import sys

is_prime()
is_lower()

CustomerOrder
InvoiceInfo

Constants are all capital letters
MAX_COUNT
PI_VALUE
====

1. All variables are references to Objects
2. All identifiers are variables
a. Function name
b. Class name
c. Module name
d. Other variable names

3. All assignments are "assign by references"
4. Every object is uniquely identified by the object-id.
=====

numeric operater
+
-
*
/
%

**   power
//   floordiv
====

Object types
Immutable objects
Hashable objects
Iterable objects
Collection
-> Sequence : An ordered collection of items
str (string)
unicode
bytearray
tuple
=====

references
Variables
Attributes of an object
Index of a sequence
Key of a mapping
:
====

sequence operator
+    Concatenation
*    Repetition
%    String format operator
====

sequence function
sum()
max(), min(), all(), any()

sorted(), reversed()

enumerate(), zip(), itertools.izip()

iter()

len()
====

sequences
Sequences -> str, unicode, tuple, list, bytearray, buffer
Sequence operations
-------------------
+  concatenation
a = 10, 20, 30
b = 40, 50, 60
c = a + b
print c
10, 20, 30, 40, 50, 60

* repetition
a = 10, 20
b = a * 5
print b
10, 20, 10, 20, 10, 20, 10, 20 ...

[] indexing
a
a[-1]
a[2:3]
a[::2]

Sequence methods
----------------
a.index(element)
a.count(element)

General sequence functions
--------------------------
len(s)
min(s)
max(s)
all(s)
any(s)

sum(s)

zip(s1, s2)
enumerate(s)

reversed(s)
sorted(s)
=====

statements
1. print tuple          # Prints each element of a tuple as a string
print "Hello world", "another string", 10, 5.6

2. Assignments
a = 10               # simple assignment
a = 10, 20           # tuple packing
b, c = a             # tuple unpacking
a, b = 10, 20        # parallel assignment
a += 5               # augmented assignment

3. del variable_or_ref  # Remove a variable or a reference
del a                # Remove the variable 'a' and
# decrement the reference count
# of the object referred by 'a'

4. import module        # Load a module and in the module namespace

import sys           # Load a module called 'sys'
sys.version          # Access members of 'sys' module using 'sys' namespace

import sys as system # Load a module called 'sys' under 'system' namespace
system.version       # Access member called 'version' from
# 'system' namespace

5. from module import member  # Load a module and make members of that module
# accessible in the current namespace

from sys import getsizeof  # getsizeof() function can now be accessed
# directly without namespace resolution.

from sys import getsizeof as sizeof    # import a member named 'getsizeof'
# from module named 'sys' as a
new name 'sizeof'

6. pass                 # Represent a do-nothing statement

7. raise SomeError      # Raise a runtime exception of type SomeError

8. assert expression    # Assert expression to be true
-------------------------------------------------------------------------------
Other simple statements
-----------------------
9.  return value        # Return a value from a function
10. break               # Break out of a while/for loop
12. yield value         # Yield a value to the generator
13. global variable     # Declare a variable in a function to be global
===

special methods
-   __sub__
*   __mul__
/   __div__
**  __pow__
//  __floordiv__
%   __mod__

<<  __lshift__
>>  __rshift__
|   __or__
&   __and__
^   __xor__
~   __neg__
-= __isub__
....
a == b
a.__cmp__(b) == 0

a > b
a.__cmp__(b) > 0

a < b
a.__cmp__(b) < 0
====

strings
a = "He said - I'll be back"
a = 'He said "I will be back" and we waited'
a = "He said - \"I'll be back\" and we waited"
a = 'He said - "I\'ll be back" and we waited'

a = """He said - "I'll be back" and we waited"""

install_path = r'C:\backup\remote\network\temp\assorted'
unicode_data = u'Hello world'
---------------------------------------------------------
String operators
----------------
%  - String format operator
+  - String concatenation (from sequence)
*  - String repetition (from sequence)
[] - String index operator (from sequence)
====

oop test

from __future__ import print_function

class Car:
def __init__(self, name):
self.name = name

@staticmethod
def sell():
print("Sold this car...")

@classmethod
def count(c):
print("c =", c)

def drive(self):
print("Driving", self.name)

c = Car("Honda")
c.drive()
c.sell()
Car.sell()
c.count()
Car.count()

Driving Honda
Sold this car...
Sold this car...
c =
c =
=====

Performance check

"""
A simple python program to test CPU performance by
generating a series of prime numbers
"""
from __future__ import print_function
import sys
if sys.version_info == 2:
from __builtin__ import xrange as range

NUM_PRIMES = 100000

def is_prime(number):
"Returns True if 'number' is prime"
limit = int(number ** 0.5) + 1
for i in range(2, limit):
if number % i == 0:
return False
return True

def gen_prime(num):
"Generates first 'num' series of prime numbers"
i = 2
while num:
if is_prime(i):
print(i)
num -= 1
i += 1

from time import time

print("Generating %d prime numbers..." % NUM_PRIMES)
start = time()
gen_prime(NUM_PRIMES)
duration = time() - start
print("gen_prime(%d) took %f seconds" % (NUM_PRIMES, duration))

gen_prime(100000) took 12.242300 seconds
=====

list vs tuple vs set
from timeit import timeit
from sys import argv

sample_size = int(argv) if len(argv) > 1 else 1000

num_iters = 100000
datatypes = "tuple", "list", "set"

setup_code = """
from random import randint
a = {datatype}(range({limit}))
"""
stmt_code = "randint(0, {limit}) in a"

for dt in datatypes:
duration = timeit(
setup=setup_code.format(datatype=dt, limit=sample_size),
stmt=stmt_code.format(limit=sample_size),
number=num_iters)

print("Searching for a random element in {} sized {} took {} seconds".format(
sample_size, dt, duration))

##output
Searching for a random element in 1000 sized tuple took 0.7152920619992074 seconds
Searching for a random element in 1000 sized list took 0.737770362989977 seconds
Searching for a random element in 1000 sized set took 0.14120669901603833 seconds                        ====

from __future__ import print_function

from performance import profile_time, print_log
import sys

if sys.version_info == 2:
range = xrange

def worker(w):
y = 0
print("Starting worker: {}\n".format(w), end="")
for i in range(1000):
for j in range(10000):
y += i*j
print("Worker {} complete: {}\n".format(w, y), end="")

@profile_time
def start_workers():
pool = { }
for i in range(16):
pool[i].start()
print("Created 16 workers...\n", end="")

for i in range(16):
pool[i].join()
print("All workers complete...\n", end="")

start_workers()

print_log()
===

from __future__ import print_function
import struct

silly_data_format = 'ich64s'
with open("silly_data.dat", "rb") as source:
print(data)

#output
(42, b'a', 67, b"hello world from the 'C'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00")
=====

while True:
line = input("Enter a line: ")
print(line)
===

local vs global
a = 10
def foo():
a = 20
print("In foo: a =", a)
print(globals())
print("-" * 30)
print(locals())
globals()['a'] = 40

foo()
print("In main: a =", a)

##output
In foo: a = 20
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external .sourcefileloader="" 0x10821f780="" at="" object="">, '__spec__': None, '__annotations__': {}, '__builtins__': , '__file__': 's1.py', '__cached__': None, 'a': 10, 'foo': }
------------------------------
{'a': 20}
In main: a = 40
====

##scope oo
color = "white"

def start():
print("Started program...")

class Car:
color = "red"

def __init__(self):
self.color = "green"

def start(self):
print("Started car...")

def drive(self):
start()
print("Driving a", color, "car")
print("Car.color =", Car.color)
print("self.color =", self.color)

c = Car()
c.drive()

##output
Started program...
Driving a white car
Car.color = red
self.color = green
=====

a = 10

def foo():
print("In foo: a =", a)
a = 40

foo()
print("In main: a =", a)

##output
Traceback (most recent call last):
File "scope.py", line 7, in
foo()
File "scope.py", line 4, in foo
print("In foo: a =", a)
UnboundLocalError: local variable 'a' referenced before assignment
====

from __future__ import print_function

a = 10

def foo():
global a
print("In foo: a =", a)
a = 20

foo()
print("In main: a =", a)

##output
In foo: a = 10
In main: a = 20
===

a = [10, 20, 30]

def foo():
print("In foo: a =", a)
a = 100

foo()
print("In main: a =", a)

##output
In foo: a = [10, 20, 30]
In main: a = [100, 20, 30]
====

color = "white"

def start():
print("Starting program...")

class Car:
color = "blue"

def __init__(s):
s.color = "green"

def start(self):
print("Starting car...")

def drive(self):
self.start()
print("Driving a", color, "car")
print("Car.color =", Car.color)
print("self.color =", self.color)

c = Car()
c.drive()

print(color)
print(Car.color)
print(c.color)

##output
Starting car...
Driving a white car
Car.color = blue
self.color = green
white
blue
green
=====

a = [10, 20, 30]
b = [40, 50, 60]

from itertools import chain

for i in chain(a, b):
print(i)

##output
10
20
30
40
50
60
====

#print table

num = int(input("Enter a number: "))
for i in 1, 2, 3, 4, 5, 6, 7, 8, 9, 10:
print(num, "x", i, "=", num*i)

##output
Enter a number: 5
5 x 1 = 5
5 x 2 = 10
5 x 3 = 15
5 x 4 = 20
5 x 5 = 25
5 x 6 = 30
5 x 7 = 35
5 x 8 = 40
5 x 9 = 45
5 x 10 = 50
====

print("Hello world")
name = "John"
print("Name is {}".format(name))
print("Square of 2 is", square(2))

def square(x): return x*x
===

from foo import square
print("square of 9 is", square(9))
====

color = "white"

class Car:
color = "red"

def __init__(self):
self.color = "green"

def drive(self):
print("Driving a", color, "car")
print("Car.color =", Car.color)
print("self.color =", self.color)

c = Car()
c.drive()

Driving a white car
Car.color red
self.color =  green
=====

chaining
a = [10, 20, 30, 40]
b = [50, 60, 70, 80]
from itertools import chain

for i in chain(a, b): print i
====

cyclic reference

class Car:
def __init__(self):
print("Car object created...")

def __del__(self):
print("Car object destroyed...")

class Person:
def __init__(self):
print("Person object created...")

def __del__(self):
print("Person object destroyed...")

p = Person()
c = Car()
c.owner = p
p.owns = c

print("Created car and person...")
del c
del p
print("Variables c and p deleted...")
import gc
print(gc.collect())

##output
Person object created...
Car object created...
Created car and person...
Variables c and p deleted...
Person object destroyed...
Car object destroyed...
4
=========

delete

class Person:

def __init__(self):
print("User object created...")

def __del__(self):
print("User object destroyed...")

class Car:

def __init__(self):
print("Car object created...")

def __del__(self):
print("Car object destroyed...")

p = Person()
c = Car()

p.owns = c
c.owner = p

print("Car and Person object created...")

#output
User object created...
Car object created...
Car and Person object created...
User object destroyed...
Car object destroyed...
=======

dict:
user_info = dict(name="John",
dept="IT",
city="Bengaluru")

for key, value in user_info.items():
print("{} -> {}".format(key, value))

#or
user_info = {"name": "nawraj","role": "Engineer","dept": "Eng"}

for key, value in user_info.items():
print("{} -> {}".format(key, value))

=====

duck typing

a = [10, 20, 30]
b = (10, 20, 30)

def c():
print("Hello world")

class d:
def __init__(self):
print("Created an object...")

objs = a, b, c, d

print(objs)

if callable(t): t()

run(objs)

#output
([10, 20, 30], (10, 20, 30), , )
Hello world
Created an object...
====

findall

"""
Preliminary exercise
====================
Implement the findall() function below that must return a list
of indices all occurrence of a substring found in a string
(both passed as arguments to the function).

Example usage:
--------------
>>> quote = '''
... When I see a bird
... that walks like a duck
... and swims like a duck
... and quacks like a duck,
... I call that bird a duck
... '''

>>> findall(quote, "duck")
[37, 59, 82, 107]

"""

def findall(main_string: str, sub_string: str) -> list:
"""
Returns a list of indices of each occurrence of
sub_string in main_string

Example usage:
--------------
>>> poem = '''
... A fly and flea flew into a flue,
... said the fly to the flea 'what shall we do ?'
... 'let us fly' said the flea
... and said the fly 'let us flee'
... and so they flew through a flaw in to the flue.
... '''

>>> findall(poem, 'fly')
[3, 43, 88, 120]

"""
indices = []
i = 0
while True:
i = main_string.find(sub_string, i)
if i == -1: break
indices.append(i)
i += len(sub_string)
return indices

if __name__ == '__main__':
import doctest
doctest.testmod()
# Running this program using 'python3 findall.py' should
# ideally pass all tests in the doctest - on implementation
# of findall() function
======

name = "smith"

class User:
"""
A simple User class

eoiwru iowe uriowue rio wuerio
sdlj fskldj fklsdj flksdjfkls

"""
def __init__(self, name):
"""
Create a new user object with 'name' passed
as argument.
"""
self.name = name

def greet(self): print(self.name, "says Hello!")

def welcome(): print("Welcome to Python...")

def square(x):
"Returns square of argument x."
return x*x

if __name__ == '__main__':
print("Main program running...")
print("a =", a)
print("name =", name)
print("square(2) =", square(2))
u = User("John")
u.greet()

##otuput
Main program running...
a = 100
name = smith
square(2) = 4
John says Hello!
=========

for loop implementation

a = [10, 20, 30, 40, 50]

#for i in a: print i,

iterator = iter(a)
if hasattr(a, '__getitem__') and hasattr(a, '__len__'):
length = len(a)
index = 0
while index < length:
i = a[index]
index += 1

if hasattr(a, '__len__'):  # For fixed-length iterable objects
length = len(a)        # (a.k.a collections/containers)
while length:
i = next(iterator)
print (i),  # 'for' loop body
length -= 1

else:                      # For generators/views/non-collection iterables
try:
while True:
i = next(iterator)
print (i),  # 'for' loop body
except StopIteration:
pass

#output
10
20
30
40
50
========

class Car:
def __init__(self):
print("Created a new car object...")

def __del__(self):
print("Car object is being destroyed!")

c1 = Car()
c2 = c1
c3 = c2
print(c1, c2, c3)

print("Changing c2...")
c2 = 100

print("Deleting c1...")
del c1

print("Changing c3...")
c3 = "Hello world"

print("End of program...")

##output
Created a new car object...
<__main__ .car="" 0x109a13780="" at="" object=""> <__main__ .car="" 0x109a13780="" at="" object=""> <__main__ .car="" 0x109a13780="" at="" object="">
Changing c2...
Deleting c1...
Changing c3...
Car object is being destroyed!
End of program...
=====

a = "10"

def foo():
from builtins import int
la = a

for i in range(1000000):
print("Value of a is", int(la))

foo()
========

def greet():
print("Hello world")

def greet(user):
print("Hello", user)

greet("John")
greet()

##output
Hello John
Traceback (most recent call last):
greet()
TypeError: greet() missing 1 required positional argument: 'user'
====

def greet(user=None):
if user is None:
print("Hello world")
else:
print("Hello", user)

greet("John")
greet()

Hello John
Hello world

========
use of sep

print("Hello world", "Another string", sep="")

## sep is the separator used between multiple values when printing. The default is a space (sep=' ')
===

use fo end

print("Hello world", end="\t")

# Python’s print() function comes with a parameter called ‘end’. By default, the value of this parameter is ‘\n’, i.e. the new line character. You can end a print statement with any character/string using this parameter
===

Infinite loop
from itertools import count
for i in count(10, 2):
print(i)

# count(start=0, step=1) --> count object

from itertools import count

i = 0
while True:
print(i)
i += 1

for i in count():
print(i)
====

module test

def foo():
from time import ctime
print("Time now is", ctime())

def bar():
from time import ctime
print("Time now in bar is", ctime())

foo()
bar()

Time now is Fri Oct 20 10:12:17 2017
Time now in bar is Fri Oct 20 10:12:17 2017

## ctime : Convert a time in seconds since the Epoch to a string in local time.
This is equivalent to asctime(localtime(seconds)). When the time tuple is
not present, current time as returned by localtime() is used.

#### 1 comment:

1. I like your blog, I read this blog please update more content on hacking, further check it once at python online course