<kaggle_start><data_title>Google Play Store Apps<data_description>### Context

While many public datasets (on Kaggle and the like) provide Apple App Store data, there are not many counterpart datasets available for Google Play Store apps anywhere on the web. On digging deeper, I found out that iTunes App Store page deploys a nicely indexed appendix-like structure to allow for simple and easy web scraping. On the other hand, Google Play Store uses sophisticated modern-day techniques (like dynamic page load) using JQuery making scraping more challenging.


### Content

Each app (row) has values for catergory, rating, size, and more.<data_name>google-play-store-apps
<code># Teaching Python Programming Language For Data Scientists
# Introduction
#
# What is Python Programming Language?
# Why Python Programming Language?
# Required Libraries
#
# Python for Beginners
#
# Python Installation
# Maths Operations
# Strings Operation
# Lists Operation
# if-elif-else
# While Statement
# For Statement
# Range
# Break and Continue
# Introduction to Functions
# Lambda Expressions
# Documentation Strings
# Data Structure
# Dictionaries
# Modules
# Input and Output
# Reading and Writing Files
# Errors and Exceptions
# Classes
#
# Data Preparation and Processing
#
# Pandas
#
# Create Data
# Get Data
# Prepare Data
# Analyze Data
# Presedent Data
#
#
# NumPy
#
# The Basics
# Array Creation
# Printing Arrays
# Basic Operations
# Universal Functions
# Indexing, Slicing and Iterating
# Shape Manipulation
# Deep Copy
#
#
#
# Feature Work
# References
# Conclusion
# if you like it,please UPVOTE
# Last Update : 12.05.2019
# Introduction
# What is Python Programming Language?
# Python is the programming language. Kdnuggets is the most used programming language in data science with 65.5% according to Analytics data. Open source is free and free software. Python is a programming language that supports the object-oriented programming paradigm. Web, mobile and socket programming etc. is a scripting language with many features. However, it can operate independently on every platform. This is an indispensable programming language for data scientists and other programmers thanks to their diversity. In addition, it has a clear and easy-to-read format compared to other programming languages. With its wide library, it helps us in all kinds of software works. All software will be written in this programming language.
# Link: https://www.python.org/downloads/
# Why Python Programming Language?
# It is one of the most used programming languages by data scientists. Because there are developers all around the world. In this way, a wide variety of operations are performed. In addition, very few transactions can be performed with very little code. More than one operation is performed.Python is a powerful programming language that is easy to learn. It has a simple but effective approach to effective high-level data structures and object-oriented programming. Python's exquisite syntax and dynamic writing, combined with its interpreted nature, has made it an ideal language for scripting and rapid application development across many platforms.
# The Python interpreter can be easily expanded with new functions and data types implemented in C or C ++ (or other languages that can be called from C). Python is also available as an extension language for customizable applications.
# Required Libraries
# Numpy
# It is a Python library that allows us to perform multiple operations on arrays, including multidimensional arrays, various derived objects (such as masked arrays and matrices) and lots of mathematical, logical, shape manipulation, sorting, sorting, discrete Fourier. Statistical procedures and simulations can be performed using NumPy. NumPy library with C ++ does not cause any loss of performance.
# Pandas
# Pandas is an open source, BSD licensed library, offering high-performance, easy-to-use data structures and data analysis tools for the Python programming language. It is a library designed to work with data, allowing you to run your non-structural data as in structural databases. Pandas is designed for fast and easy data processing, collection and visualization. Therefore, it is a very preferred library for data scientists. In addition, with its own data-editing functions, you can convert your data into any format. Thanks to Pandas, we make preliminary work for our model and make it ready for the model. As a preliminary study, it is necessary to carry out editing studies by detecting missing, slingshot, incompatible and repetitive data. In this way, you can train your model smoothly and get the results properly. To use the Pandas library, we need to upload it to Anaconda.
# Scikit-Learn
# It includes many machine learning algorithms such as classification, regression, clustering, size reduction, model selection and data removal. It is an open source library with BSD license. Together with the algorithms it contains, it creates a fast and secure coding structure. It includes the functions of expressing frequently used mathematical and physical problems in computer environment. In addition, thanks to its own web site you can look at the working principle of the algorithm you want. However, you can learn the mathematical infrastructure of the algorithm. In this way, according to your own data set you can develop the mathematical model of the algorithm yourself. By applying the model you have made on the data set, results can be obtained. Which of these results is effective in the mathematical model you will make out. In addition, you must install the scikit-learn library to Anaconda to use it.
# Seaborn
# Seaborn is a library in Python for making interesting and informative statistical graphs. Built on Matplotlib, numpy has been tightly integrated with the PyData stack, providing support for statistical templates from scipy, statsmodel, pandas data structures. It is used for graphs depicting general distributions that summarize data as in heat maps. Thanks to the many methods included in the content of high-level analysis can be made. However, the user can perform analyzes in interactive and animation structure. Today's data scientists put the seaborn library in place where the matplotlib library is missing. Thanks to Seaborn, interactive plots can be removed. In this way, we will show the dimensions of inter-feature interaction more clearly. Thanks to the different types of plots within itself, we are able to make analyzes in every format.
# Matplotlib
# Matplotlib graphical drawing package is one of the most important tools of scientific programming. With Matplotlib, a very powerful package, data can be visualized interactively, and high quality printouts are available for publishing. Both two- and three-dimensional graphics can be created. However, it is very simple to use. In addition, there are a lot of information about this package on the internet, but for data scientists can take all kinds of analysis. As a result of your analysis, you can save the resulting graph as output. You can then add it as an addition to the report you made later. It will be used to explain the results obtained from various analyzes and evaluations in the application orally and graphically. So much so that we use the algorithm to show us the most accurate results. In this way, we will guide us for further analysis. In addition, our corporate website helps us for all the plots we will use.
# Scipy
# SciPy refers to several related but distinct entities:The SciPy ecosystem, a collection of open source software for scientific computing in Python.The community of people who use and develop this stack.Several conferences dedicated to scientific computing in Python - SciPy, EuroSciPy and SciPy.in.The SciPy library, one component of the SciPy stack, providing many numerical routines.
# Python for Beginners
# Python Installation
# Link: https://www.youtube.com/watch?v=dX2-V2BocqQ
# There is very good expression in this section.
# Maths Operations
# Python can do a variety of mathematical operations. In this case, we do not need to define any variables. This is described in the examples below for this situation.
# The following mathematical operations are given 3 kinds of examples. In the first example, only data from the variable definition are given. Thus, results were obtained. In the second example, a variable is defined. In this example, the operations were performed. In the last example, a decimal data is obtained by dividing int data with decimal data.**
# Before continue with pandas, we need to learn logic, control flow and filtering.
# Comparison operator: ==, , <=
# Boolean operators: and, or ,not
# Filtering pandas
# Comparison operator
print(3 > 2)
print(5 < 4)
print("Apple" in ["App", "Appl", "Apple"])
print(3 != 2)
# Boolean operators
print(True and False)
print(True or False)
# Example 1
print(2 + 2)
# Example 2
a = (50 - 5 * 6) / 4
print(a)
# Example 3
print(8 / 5.0)
# Example 4
print(10 + 80)
print(10 - (25 - 85) + 45)
print(5 * 6 - (2 - 8))
# Example 5
print(17 // 3.0)  # explicit floor division discards the fractional part
# Example 6
print(18 // 4.0)
# The // operator is also provided for doing floor division no
# matter what the operands are.
# Example 7
print(17 % 3)
# Example 6 Mode operation; between the first number and the second number.
# As a result of this operation, the remaining number is obtained.
# With Python, it is possible to use the ** operator to calculate powers
# Example 8
print(5**2)  # 5 squared
# Example 9
print(2**8)  # 2 to power of 8
# The equal sign (=) is used to assign a value to a variable.
# Example 10
width = 8
height = 10
print(width * height)

# Strings Operation
# Besides numbers, Python can also manipulate strings, which can be expressed in several ways. They can be enclosed in single quotes ('...') or double quotes ("...") with the same result. \ can be used to escape quotes:
s1 = "string"
s2 = "data"
print(s1 + " " + s2)
# Strings can be concatenated (glued together) with the
# + operator, and repeated with *:
index = 3
print("un " * index)
# Two or more string literals
print("Py" "thon")
prefix = "Py"
# prefix='thon'
prefix + "thon"
# Strings can be indexed (subscripted)
word = "Python"
print(word[0])  # character in position 0
print(word[5])  # character in position 5
print(word[-1])  # last character
print(word[-2])  # second-last character
# In addition to indexing, slicing is also supported.
# While indexing is used to obtain individual characters,
# slicing allows you to obtain a substring:
print(word[0:2])  # characters from position 0 (included) to 2 (excluded)
print(word[2:5])  # characters from position 2 (included) to 5 (excluded)
# Note how the start is always included, and the end always
# excluded. This makes sure that s[:i] + s[i:] is always equal to s:
print(word[:2] + word[2:])
print(word[:4] + word[4:])
# Slice indices have useful defaults; an omitted first index defaults
# to zero, an omitted second index defaults to the size of the
# string being sliced.
print(word[:2])  # character from the beginning to position 2 (excluded)
print(word[4:])
print(word[-2:])
print(word[1:3])  # character from the beginning to position 3 (excluded)
print("J" + word[1:])
print(word[:2] + "py")

# Lists Operation
# Python knows a number of compound data types, used to group together other values. The most versatile is the list, which can be written as a list of comma-separated values (items) between square brackets. Lists might contain items of different types, but usually the items all have the same type.
squares = [1, 4, 9, 16, 25]
squares
# Like strings (and all other built-in sequence type),
# lists can be indexed and sliced:
print(squares[0])  # indexing returns the item
print(squares[-1])
print(squares[-3:])  # slicing returns a new list
print(squares[:])  # all
# All slice operations return a new list containing the requested
# elements. This means that the following slice returns a new
# (shallow) copy of the list:
squares + [36, 49, 64, 81, 100]
cubes = [1, 8, 27, 65, 125]
4**3
cubes[3] = 64  # replace the wrong value
cubes
# You can also add new items at the end of the list, by using the
# append() method (we will see more about methods later):
cubes.append(216)  # add the cube of 6
cubes.append(7**3)  # and the cube of 7
cubes
# Assignment to slices is also possible, and this can even change
# the size of the list or clear it entirely:
letters = ["a", "b", "c", "d", "e", "f", "g"]
print(letters)
# replace some values
letters[2:5] = ["C", "D", "E"]
print(letters)
# now remove them
letters[2:5] = []
letters
# clear the list by replacing all the elements with an empty list
letters[:] = []
letters
# The built-in function len() also applies to lists:
letters = ["a", "b", "c", "d"]
len(letters)

# if-elif-else
# Decision making is required when we want to execute a code only if a certain condition is satisfied.
# [i + 1 for i in num1 ]: list of comprehension
# i +1: list comprehension syntax
# for i in num1: for loop syntax
# i: iterator
# num1: iterable object
# The if…elif…else statement is used in Python for decision making.
# Perhaps the most well-known statement type is the if statement. For example:
# We use list comprehension for data analysis often.
# list comprehension: collapse for loops for building lists into a single line
# Ex: num1 = [1,2,3] and we want to make it num2 = [2,3,4]. This can be done with for loop. However it is  unnecessarily long. We can make it one line code that is list comprehension.
# Example of list comprehension
num1 = [1, 2, 3]
num2 = [i + 1 for i in num1]
print(num2)
# Conditionals on iterable
num1 = [5, 10, 15]
num2 = [i**2 if i == 10 else i - 5 if i < 7 else i + 5 for i in num1]
print(num2)
# If the number is positive, we print an appropriate message
num = 3
if num > 0:
    print(num, " is a positive number")
print("This is always printed")
num = -1
if num > 0:
    print(num, " is a positive number")
print("This is also always printed")
# Program checks if the number is positive or negative
# And displays an appropriate message
num = 3
# Try these two variations as well.
# num = -5
# num = 0
if num >= 0:
    print("Positive or Zero")
else:
    print("Negative number")
# In this program,
# we check if the number is positive or
# negative or zero and
# display an appropriate message
num = 3.4
# Try these two variations as well:
# num = 0
# num = -4.5
if num > 0:
    print("Positive number")
elif num == 0:
    print("Zero")
else:
    print("Negative number")
# In this program, we input a number
# check if the number is positive or
# negative or zero and display
# an appropriate message
# This time we use nested if
num = float(input("Enter a number: "))
if num >= 0:
    if num == 0:
        print("Zero")
    else:
        print("Positive number")
else:
    print("Negative number")
var = 100
if var == 100:
    print("Value of expression is 100")
print("Good bye!")

# While
# With the while loop we can execute a set of statements as long as a condition is true.
# Stay in loop if condition( i is not equal 5) is true
i = 0
while i != 5:
    print("i is: ", i)
    i += 1
print(i, " is equal to 5")
# Stay in loop if condition( i is not equal 5) is true
lis = [1, 2, 3, 4, 5]
for i in lis:
    print("i is: ", i)
print("")
# Enumerate index and value of list
# index : value = 0:1, 1:2, 2:3, 3:4, 4:5
for index, value in enumerate(lis):
    print(index, " : ", value)
print("")
# For dictionaries
# We can use for loop to achive key and value of dictionary. We learnt key and value at dictionary part.
dictionary = {"spain": "madrid", "france": "paris"}
for key, value in dictionary.items():
    print(key, " : ", value)
print("")
i = 1
while i < 6:
    print(i)
    i += 1
# With the break statement we can stop the loop even if the while condition is true:
i = 1
while i < 6:
    print(i)
    if i == 3:
        break
    i += 1
# With the continue statement we can stop the current iteration, and continue with the next:
i = 0
while i < 6:
    i += 1
    if i == 3:
        continue
    print(i)

# For Statements
# The for statement in Python differs a bit from what you may be used to in C or Pascal. Rather than always iterating over an arithmetic progression of numbers (like in Pascal), or giving the user the ability to define both the iteration step and halting condition (as C), Python’s for statement iterates over the items of any sequence (a list or a string), in the order that they appear in the sequence. For example (no pun intended):
words = ["cats", "window", "defenstrate"]
for w in words:
    print(w, len(w))

# If you need to modify the sequence you are iterating over while inside the loop (for example to duplicate selected items), it is recommended that you first make a copy. Iterating over a sequence does not implicitly make a copy. The slice notation makes this especially convenient:
for w in words[:]:
    if len(w) > 6:
        words.insert(0, w)
for i, w in enumerate(words):
    print((i + 1), ":", w)
# Loop through the letters in the word "banana":
for x in "banana":
    print(x)

# Range
# If you do need to iterate over a sequence of numbers, the built-in function range() comes in handy. It generates lists containing arithmetic progressions:
# To loop through a set of code a specified number of times, we can use the range() function,
# The range() function returns a sequence of numbers, starting from 0 by default, and increments by 1 (by default), and ends at a specified number.
for i in range(10):
    print(i)

# The given end point is never part of the generated list; range(10) generates a list of 10 values, the legal indices for items of a sequence of length 10. It is possible to let the range start at another number, or to specify a different increment (even negative; sometimes this is called the ‘step’):
for i in range(5, 10):
    print(i)
print("#" * 30)
for i in range(0, 10, 3):  # 0 start 10 finish 3 step
    print(i)
print("#" * 30)
for i in range(-10, -100, -30):  # 0 start 10 finish 3 step
    print(i)
a = ["Mary", "had", "a", "little", "lamb"]
for i in range(len(a)):
    print(i, a[i])
import numpy as np

new_list = np.arange(1, 20)
for i in np.arange(1, 30, 2):
    print(i)
# The else keyword in a for loop specifies a block of code to be executed when the loop is finished:
for x in range(6):
    print(x)
else:
    print("Finish")

# Break and Continue
# The break statement, like in C, breaks out of the innermost enclosing for or while loop.
# Loop statements may have an else clause; it is executed when the loop terminates through exhaustion of the list (with for) or when the condition becomes false (with while), but not when the loop is terminated by a break statement. This is exemplified by the following loop, which searches for prime numbers:
for n in range(2, 10):
    for x in range(2, n):
        if n % x == 0:
            print(n, " equal", x, "*", n / x)
            break
    else:
        # loop fell through without finding a factor
        print(n, " is a prime number")

# (Yes, this is the correct code. Look closely: the else clause belongs to the for loop, not the if statement.)
# When used with a loop, the else clause has more in common with the else clause of a try statement than it does that of if statements: a try statement’s else clause runs when no exception occurs, and a loop’s else clause runs when no break occurs. For more on the try statement and exceptions, see Handling Exceptions.
# The continue statement, also borrowed from C, continues with the next iteration of the loop:
# iteration example
name = "ronaldo"
it = iter(name)
print(next(it))  # print next iteration
print(*it)  # print remaining iteration

#
# ITERATORS
# iterable is an object that can return an iterator
# iterable: an object with an associated iter() method
# example: list, strings and dictionaries
# iterator: produces next value with next() method
#
for num in range(2, 10):
    if num % 2 == 0:
        print("Found an even number", num)
        continue
    print("Found a number", num)


# Introduction to Functions
# We can create a function that writes the Fibonacci series to an arbitrary boundary:
def fib(n):
    # print a fibonacci series up to n
    a, b = 0, 1
    while a < n:
        print(a)
        a, b = b, a + b


fib(2000)


def my_function1(country="Turkey"):
    print("I am from " + country)


my_function1()
my_function1("Canada")


# The keyword def introduces a function definition. It must be followed by the function name and the parenthesized list of formal parameters. The statements that form the body of the function start at the next line, and must be indented.
# The most useful form is to specify a default value for one or more arguments. This creates a function that can be called with fewer arguments than it is defined to allow. For example:


def ask_ok(prompt, retries=4, complaint="Yes or no, please!"):
    while True:
        ok = raw_input(prompt)
        if ok in ("y", "ye", "yes"):
            return True
        if ok in ("n", "no", "nop", "nope"):
            return False
        retries = retries - 1
        if retries < 0:
            raise IOError("refusenik user")
        print(complaint)


# This function can be called in several ways:
# giving only the mandatory argument: ask_ok('Do you really want to quit?')
# giving one of the optional arguments: ask_ok('OK to overwrite the file?', 2)
# or even giving all arguments: ask_ok('OK to overwrite the file?', 2, 'Come on, only yes or no!')
# The default values are evaluated at the point of function definition in the defining scope, so that
i = 5


def f(arg=i):
    print(arg)


i = 6
f()


# Important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. For example, the following function accumulates the arguments passed to it on subsequent calls:
def f(a, L=[]):
    L.append(a)
    return L


print(f(1))
print(f(2))
print(f(3))


# If you don’t want the default to be shared between subsequent calls, you can write the function like this instead:
def f(a, L=None):
    if L is None:
        L = []
    L.append(a)
    return L


f([5, 4, 3])


# Functions can also be called using keyword arguments of the form kwarg=value. For instance, the following function:
def parrot(voltage, state="a stiff", action="voom", type="Norwegian Blue"):
    print(
        "-- This parrot wouldn't",
        action,
    )
    print("if you put", voltage, "volts through it.")
    print("-- Lovely plumage, the", type)
    print("-- It's", state, "!")


# accepts one required argument (voltage) and three optional arguments (state, action, and type). This function can be called in any of the following ways:
# accepts one required argument (voltage) and three optional arguments (state, action, and type). This function can be called in any of the following ways:
parrot(1000)  # 1 positional argument
parrot(voltage=1000)  # 1 keyword argument
parrot(voltage=1000000, action="VOOOOOM")  # 2 keyword arguments
parrot(action="VOOOOOM", voltage=1000000)  # 2 keyword arguments
parrot("a million", "bereft of life", "jump")  # 3 positional arguments
parrot("a thousand", state="pushing up the daisies")  # 1 positional, 1 keyword


# When a final formal parameter of the form **name is present, it receives a dictionary (see Mapping Types — dict) containing all keyword arguments except for those corresponding to a formal parameter. This may be combined with a formal parameter of the form *name (described in the next subsection) which receives a tuple containing the positional arguments beyond the formal parameter list. (*name must occur before **name.) For example, if we define a function like this:
def cheeseshop(kind, *arguments, **keywords):
    print("-- Do you have any", kind, "?")
    print("-- I'm sorry, we're all out of", kind)
    for arg in arguments:
        print(arg)
    print("-" * 40)
    keys = sorted(keywords.keys())
    for kw in keys:
        print(kw, ":", keywords[kw])


"""cheeseshop("Limburger", "It's very runny, sir.",
           "It's really very, VERY runny, sir.",
           shopkeeper='Michael Palin',
           client="John Cleese",
           sketch="Cheese Shop Sketch")"""


# The reverse situation occurs when the arguments are already in a list or tuple but need to be unpacked for a function call requiring separate positional arguments. For instance, the built-in range() function expects separate start and stop arguments. If they are not available separately, write the function call with the *-operator to unpack the arguments out of a list or tuple:
print(list(range(3, 6)))  # normal call with separate arguments
args = [3, 6]
list(range(*args))  # call with arguments unpacked from a list


def tri_recursion(k):
    if k > 0:
        result = k + tri_recursion(k - 1)
        print(result)
    else:
        result = 0
    return result


tri_recursion(6)


# Lambda Expressions
# Small anonymous functions can be created with the lambda keyword. This function returns the sum of its two arguments: lambda a, b: a+b. Lambda functions can be used wherever function objects are required. They are syntactically restricted to a single expression. Semantically, they are just syntactic sugar for a normal function definition. Like nested function definitions, lambda functions can reference variables from the containing scope:
def make_incrementor(n):
    return lambda x: x + n


f = make_incrementor(50)
print(f(2))
print(f(1))


# The above example uses a lambda expression to return a function. Another use is to pass a small function as an argument:
pairs = [(1, "one"), (2, "two"), (3, "three"), (4, "four")]
pairs.sort(key=lambda pair: pair[1])
pairs
# A lambda function that adds 10 to the number passed in as an argument, and print the result:
x = lambda a: a + 10
print(x(5))
x = 5
y = lambda x: x**2
y(10)

# A lambda function that adds 10 to the number passed in as an argument, and print the result:
x = lambda a: a + 120
print(x(15))

# A lambda function that multiplies argument a with argument b and print the result:
x = lambda a, b, c: a + b + c
print(x(5, 6, 2))
number_list = [1, 2, 3]
y = map(lambda x: x**2, number_list)
print(list(y))


# The power of lambda is better shown when you use them as an anonymous function inside another function.
# Say you have a function definition that takes one argument, and that argument will be multiplied with an unknown number:
def myfunc(n):
    return lambda a: a * n


mydoubler = myfunc(2)
print(mydoubler(11))


# Documentation Strings
# There are emerging conventions about the content and formatting of documentation strings.
# The first line should always be a short, concise summary of the object’s purpose. For brevity, it should not explicitly state the object’s name or type, since these are available by other means (except if the name happens to be a verb describing a function’s operation). This line should begin with a capital letter and end with a period.
# If there are more lines in the documentation string, the second line should be blank, visually separating the summary from the rest of the description. The following lines should be one or more paragraphs describing the object’s calling conventions, its side effects, etc.
# Here is an example of a multi-line docstring:
def my_fucntion():
    """Do nothing, but document it
    No,really, it doesn't do anything.
    """
    pass


print(my_fucntion)


# Data Structures
# This chapter describes some things you’ve learned about already in more detail, and adds some new things as well.
a = [66.25, 333, 333, 1, 12, 154, 789]
print(a.count(333), a.count(66.25), a.count("x"))
a.insert(2, -1)
a.append(333)
print(a)
a.index(333)
a.remove(333)
a.reverse()
print(a)
a.sort()
print(a.pop())
a

# The list methods make it very easy to use a list as a stack, where the last element added is the first element retrieved (“last-in, first-out”). To add an item to the top of the stack, use append(). To retrieve an item from the top of the stack, use pop() without an explicit index. For example:
stack = [3, 4, 5]
stack.append(6)
stack.append(7)
print(stack)
print(stack.pop())
print(stack.pop())
print(stack)


def f(x):
    return x % 3 == 0 or x % 5 == 0


f(9)


# map(function, sequence) calls function(item) for each of the sequence’s items and returns a list of the return values. For example, to compute some cubes:
def cube(x):
    return x * x * x


cube(5)


# More than one sequence may be passed; the function must then have as many arguments as there are sequences and is called with the corresponding item from each sequence (or None if some sequence is shorter than another). For example:
seq = range(1, 8)
list(seq)


def add(x, y):
    return x + y


add(5, 7)


# List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.
# For example, assume we want to create a list of squares, like:
squares = []
for x in range(10):
    squares.append(x**2)
squares
for x in range(0, 18, 2):
    print(x)

# We can obtain the same result with:
squares = [i * 2 for i in range(1, 10, 1)]
squares
[(x, y) for x in [1, 2, 3] for y in [3, 1, 4] if x != y]
a = [-1, 1, 66, 45, 87, 65, 12, 65]
del a[0]
print(a)
del a[2:4]
print(a)
del a[:]
print(a)
t = 123, 45, 65, 78, "hi!"
print(t[0])
print(t)
# Tuples may be nested:
u = t, (1, 2, 3, 4, 5, 6)
u

# A special problem is the construction of tuples containing 0 or 1 items: the syntax has some extra quirks to accommodate these. Empty tuples are constructed by an empty pair of parentheses; a tuple with one item is constructed by following a value with a comma (it is not sufficient to enclose a single value in parentheses). Ugly, but effective. For example:
empty = ()
singleton = ("hello",)
len(empty)
len(singleton)
print(singleton)

# Dictionaries
# Another useful data type built into Python is the dictionary (see Mapping Types — dict). Dictionaries are sometimes found in other languages as “associative memories” or “associative arrays”. Unlike sequences, which are indexed by a range of numbers, dictionaries are indexed by keys, which can be any immutable type; strings and numbers can always be keys. Tuples can be used as keys if they contain only strings, numbers, or tuples; if a tuple contains any mutable object either directly or indirectly, it cannot be used as a key. You can’t use lists as keys, since lists can be modified in place using index assignments, slice assignments, or methods like append() and extend().
# create dictionary and look its keys and values
dictionary = {"spain": "madrid", "usa": "vegas"}
print(dictionary.keys())
print(dictionary.values())
# Keys have to be immutable objects like string, boolean, float, integer or tubles
# List is not immutable
# Keys are unique
dictionary["spain"] = "barcelona"  # update existing entry
print(dictionary)
dictionary["france"] = "paris"  # Add new entry
print(dictionary)
del dictionary["spain"]  # remove entry with key 'spain'
print(dictionary)
print("france" in dictionary)  # check include or not
dictionary.clear()  # remove all entries in dict
print(dictionary)
tel = {"jack": 4098, "sape": 4139}
tel["guido"] = 4127
print(tel)
print(tel["jack"])
del tel["sape"]
tel["irv"] = 4127
print(tel)
print(tel.keys())
print(tel.values())
"jack" in tel

# The dict() constructor builds dictionaries directly from sequences of key-value pairs:
dict([("sape", 4139), ("guido", 4127), ("jack", 4098)])

# In addition, dict comprehensions can be used to create dictionaries from arbitrary key and value expressions:
{x: x**2 for x in (2, 4, 6)}
for i, v in enumerate(["tic", "tac", "toe"]):
    print(i, v)


# Modules
# If you quit from the Python interpreter and enter it again, the definitions you have made (functions and variables) are lost. Therefore, if you want to write a somewhat longer program, you are better off using a text editor to prepare the input for the interpreter and running it with that file as input instead. This is known as creating a script. As your program gets longer, you may want to split it into several files for easier maintenance. You may also want to use a handy function that you’ve written in several programs without copying its definition into each program.
# A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module’s name (as a string) is available as the value of the global variable __name__. For instance, use your favorite text editor to create a file called fibo.py in the current directory with the following contents:
# Fibonacci numbers module
def fib(n):  # write Fibonacci series up to n
    a, b = 0, 1
    while b < n:
        print(b),
        a, b = b, a + b


def fib2(n):  # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while b < n:
        result.append(b)
        a, b = b, a + b
    return result


#
# What we need to know about functions:
# docstrings: documentation for functions. Example:
# for f():
# """This is docstring for documentation of function f"""
# tuble: sequence of immutable python objects.
# cant modify values
# tuble uses paranthesis like tuble = (1,2,3)
# unpack tuble into several variables like a,b,c = tuble
#
# How can we learn what is built in scope
import builtins

dir(builtins)


# example of what we learn above
def tuble_ex():
    """return defined t tuble"""
    t = (1, 2, 3)
    return t


a, b, c = tuble_ex()
print(a, b, c)

import sys

"""
The built-in function dir() is used to find out which names a 
module defines. It returns a sorted list of strings:
"""
print(dir(sys))

# Input and Output
# There are several ways to present the output of a program; data can be printed in a human-readable form, or written to a file for future use. This chapter will discuss some of the possibilities.
s = "Hello world"
print(str(s))
print(repr(s))
print(str(1.0 / 7.0))
x = 10 * 3.25
y = 200 * 200
s = "The value of x is " + repr(x) + ", and y is " + repr(y) + "..."
print(s)

# Reading and Writing Files
# open() returns a file object, and is most commonly used with two arguments: open(filename, mode).
# Errors and Exceptions
# Until now error messages haven’t been more than mentioned, but if you have tried out the examples you have probably seen some. There are (at least) two distinguishable kinds of errors: syntax errors and exceptions.
# while True:print('Hello World')
# Syntax errors, also known as parsing errors, are perhaps
# he most common kind of complaint you get while you are still
# earning Python:


# It is possible to write programs that handle selected exceptions. Look at the following example, which asks the user for input until a valid integer has been entered, but allows the user to interrupt the program (using Control-C or whatever the operating system supports); note that a user-generated interruption is signalled by raising the KeyboardInterrupt exception.
# while True:
# try:
# x=int(input('Please enter a number :'))
# break
# except ValueError:
# print('Stop!')
# If an exception has an argument, it is printed as the last part (‘detail’) of the message for unhandled exceptions.
# Exception handlers don’t just handle exceptions if they occur immediately in the try clause, but also if they occur inside functions that are called (even indirectly) in the try clause. For example:
def this_fails():
    x = 1 / 10


try:
    this_fails()
except ZeroDivisonError as detail:
    print("Handling run-time error :", detail)


# Classes
# Compared with other programming languages, Python’s class mechanism adds classes with a minimum of new syntax and semantics. It is a mixture of the class mechanisms found in C++ and Modula-3. Python classes provide all the standard features of Object Oriented Programming: the class inheritance mechanism allows multiple base classes, a derived class can override any methods of its base class or classes, and a method can call the method of a base class with the same name. Objects can contain arbitrary amounts and kinds of data. As is true for modules, classes partake of the dynamic nature of Python: they are created at runtime, and can be modified further after creation.
# Class objects support two kinds of operations: attribute references and instantiation.
# Attribute references use the standard syntax used for all attribute references in Python: obj.name. Valid attribute names are all the names that were in the class’s namespace when the class object was created. So, if the class definition looked like this:
class MyClass:
    """A simple example class"""

    i = 12345

    def f(self):
        return "hello world"


# Class instantiation uses function notation. Just pretend that the class object is a parameterless function that returns a new instance of the class. For example (assuming the above class):
x = MyClass()


# Of course, the __init__() method may have arguments for greater flexibility. In that case, arguments given to the class instantiation operator are passed on to __init__(). For example,
class Complex:
    def __init__(self, realpart, imagpart):
        self.r = realpart
        self.i = imagpart


x = Complex(3.0, -4.5)
print(x.i, x.r)


# Now what can we do with instance objects? The only operations understood by instance objects are attribute references. There are two kinds of valid attribute names, data attributes and methods.
# data attributes correspond to “instance variables” in Smalltalk, and to “data members” in C++. Data attributes need not be declared; like local variables, they spring into existence when they are first assigned to. For example, if x is the instance of MyClass created above, the following piece of code will print the value 16, without leaving a trace:
x.counter = 1
while x.counter < 10:
    x.counter = x.counter * 2
print(x.counter)
del x.counter


# Generally speaking, instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class:
class Dog:
    trick = []
    kind = "canine"

    def __init__(self, name):
        self.name = name

    def add_trick(self, trick):
        self.tricks.append(trick)


d = Dog("Fibo")
e = Dog("Buddy")
print(d.kind)
print(e.kind)
print(d.name)
print(e.name)


class Dog:
    tricks = []  # mistaken use of a class variable

    def __init__(self, name):
        self.name = name

    def add_trick(self, trick):
        self.tricks.append(trick)


d = Dog("Fido")
e = Dog("Buddy")
d.add_trick("roll over")
e.add_trick("play dead")
d.tricks  # unexpectedly shared by all dogs
["roll over", "play dead"]


# Data Preparation and Processing
# Pandas
# import pandas as pd
# Create Data
# The data set will consist of 5 baby names and the number of births recorded for that year (1880).
import pandas as pd  # import library as pd

names = ["Bob", "Jessica", "Mary", "Jhon", "Mel"]
births = [968, 45, 756, 12, 65]
# To merge these two lists together we will use the zip function.
BabyDataSet = list(zip(names, births))
print(BabyDataSet)

# We are basically done creating the data set. We now will use the pandas library to export this data set into a csv file.
# df will be a DataFrame object. You can think of this object holding the contents of the BabyDataSet in a format similar to a sql table or an excel spreadsheet. Lets take a look below at the contents inside df.
df = pd.DataFrame(data=BabyDataSet, columns=["Names", "Births"])
df

# Export the dataframe to a csv file. We can name the file births1880.csv. The function to_csv will be used to export the file. The file will be saved in the same location of the notebook unless specified otherwise.
# df.to_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv',index=False,header=False)
# Merge the names and the births data set using the zip function.
BabyDataSet = list(zip(names, births))
BabyDataSet[:10]
# Lets add a column
df["Year"] = 2019
# using loc accessor
df.loc[1:5, ["Names"]]
# Slicing and indexing series
df.loc[1:10, "Names":]  # 10 and "Defense" are inclusive
# replaces the deprecated ix function
# df.ix[5:,'col']
df.loc[df.index[1:], "Names"]
# replaces the deprecated ix function
# df.ix[:3,['col', 'test']]
df.loc[df.index[:3], ["Names", "Year"]]

# Get Data
# To pull in the csv file, we will use the pandas function read_csv. Let us take a look at this function and what inputs it takes.
# df=pd.read_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv')
df

# This brings us to the first problem of the exercise. The read_csv function treated the first record in the csv file as the header names. This is obviously not correct since the text file did not provide us with header names.
# To correct this we will pass the header parameter to the read_csv function and set it to None (means null in python).
##df=pd.read_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv',header=None)
print(df)
# df=pd.read_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv',names=['Name','Birth'])
print(df)
df.info()

# Info says:
# There are 999 records in the data set
# There is a column named Mary with 999 values
# There is a column named 968 with 999 values
# Out of the two columns, one is numeric, the other is non numericli>
# To actually see the contents of the dataframe we can use the head() function which by default will return the first five records. You can also pass in a number n to return the top n records of the dataframe.
df.head(4)
# Now lets take a look at the last five records of the dataframe
df.tail(4)
df.sample()

# Prepare Data
# The data we have consists of baby names and the number of births in the year 1880. We already know that we have 5 records and none of the records are missing (non-null values).
# Realize that aside from the check we did on the "Names" column, briefly looking at the data inside the dataframe should be as far as we need to go at this stage of the game. As we continue in the data analysis life cycle we will have plenty of opportunities to find any issues with the data set.
# Check data type of the columns
df.dtypes
# Check data type of births column
df.Births.dtype
# We can use the unique property of the dataframe to find all the unique records of the "Names" column.
df["Names"].unique()
# If you actually want to print the unique values:
for x in df["Names"].unique():
    print(x)
print(df["Names"].describe())

# Since we have multiple values per baby name, we need to aggregate this data so we only have a baby name appear once. This means the 1,000 rows will need to become 5. We can accomplish this by using the groupby function.
# clean name column convert to upper
df["Names"] = df.Names.apply(lambda x: x.upper())
df
# clean name column convert to upper
df["Names"] = df.Names.apply(lambda x: x.lower())
df
data_filter = data.copy()
data_filter.Genres.apply(lambda x: x.strip("&")).sample(5)
name = df.groupby("Names")
df1 = name.sum()
df1
df

# Analyze Data
# To find the most popular name or the baby name with the higest birth rate, we can do one of the following.
# Sort the dataframe and select the top row
# Use the max() attribute to find the maximum value
#
sorted = df.sort_values(["Births"], ascending=False)
sorted.head(1)
df["Births"].max()

# Present Data
# Here we can plot the Births column and label the graph to show the end user the highest point on the graph. In conjunction with the table, the end user has a clear picture that Mel is the most popular baby name in the data set.
# plot() is a convinient attribute where pandas lets you painlessly plot the data in your dataframe. We learned how to find the maximum value of the Births column in the previous section. Now to find the actual baby name of the 973 value looks a bit tricky, so lets go over it.s
# NumPy
# The Basic
# NumPy’s main object is the homogeneous multidimensional array. It is a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers. In NumPy dimensions are called axes.
import numpy as np

a = np.arange(15).reshape(3, 5)
a
a.shape
a.ndim
a.dtype.name
a.itemsize

# Array Creation
# There are several ways to create arrays.
# For example, you can create an array from a regular Python list or tuple using the array function. The type of the resulting array is deduced from the type of the elements in the sequences.
a = np.array([2, 3, 4])
a
a.dtype
b = np.array([1.2, 3.5, 5.1])
b.dtype

# A frequent error consists in calling array with multiple numeric arguments, rather than providing a single list of numbers as an argument.
# a=np.array(1,2,3,4) #that's not true
a = np.array([1, 2, 3, 4])  # that's true
# array transforms sequences of sequences into two-dimensional
# arrays, sequences of sequences of sequences into three-dimensional arrays, and so on.
b = np.array([(1.5, 2, 3), (4, 5, 6)])
b

# The function zeros creates an array full of zeros, the function ones creates an array full of ones, and the function empty creates an array whose initial content is random and depends on the state of the memory. By default, the dtype of the created array is float64.
np.zeros((3, 4))
np.eye(4, 4)
eyes = np.eye(5, 5)
np.diagonal(eyes)
array = np.array([[2, 3, 4], [6, 7, 8], [10, 11, 12]])
array.T
np.ones((2, 3, 4), dtype=np.int16)
np.empty((2, 3))

# To create sequences of numbers, NumPy provides a function analogous to range that returns arrays instead of lists.
np.arange(10, 30, 5)
np.arange(-1, 1, 0.1)
np.arange(0, 2, 0.3)

# When arange is used with floating point arguments, it is generally not possible to predict the number of elements obtained, due to the finite floating point precision. For this reason, it is usually better to use the function linspace that receives as an argument the number of elements that we want, instead of the step:
np.linspace(0, 2, 9)
np.linspace(0, 2, 10)
from numpy import pi

x = np.linspace(0, 2 * pi, 100)
f = np.sin(x)

# Printing Arrays
# When you print an array, NumPy displays it in a similar way to nested lists, but with the following layout:
# the last axis is printed from left to right,
# the second-to-last is printed from top to bottom,
# the rest are also printed from top to bottom, with each slice separated from the next by an empty line.
#
a = np.arange(6)  # 1d array
print(a)
b = np.arange(12).reshape(4, 3)  # 2d array
print(b)
c = np.arange(24).reshape(2, 3, 4)  # 3d array
print(c)
print(np.arange(10000))
print(np.arange(10000).reshape(100, 100))

# Basic Operations
# Arithmetic operators on arrays apply elementwise. A new array is created and filled with the result.
a = np.array([20, 30, 40, 50])
b = np.arange(4)
print(b)
c = a - b
print(c)
print(b**2)
print(10 * np.sin(a))
print(a < 35)

# Universal Functions
# NumPy provides familiar mathematical functions such as sin, cos, and exp. In NumPy, these are called “universal functions”(ufunc). Within NumPy, these functions operate elementwise on an array, producing an array as output.
B = np.arange(3)
print(B)
np.exp(B)
np.sqrt(B)
C = np.array([2.0, -1.0, 4.0])
np.add(B, C)
np.abs(-100)

# Indexing, Slicing and Iterating
# One-dimensional arrays can be indexed, sliced and iterated over, much like lists and other Python sequences.
a = np.arange(10) ** 3
print(a)
print(a[2])
print(a[2:5])
a[:6:2] = -1100
print(a)

# Shape Manipulation
a = np.floor(10 * np.random.random((3, 4)))
print(a)
a.shape
print(a.shape[0])

# The shape of an array can be changed with various commands. Note that the following three commands all return a modified array, but do not change the original array:
a.ravel()  # returns the array,flattend
a.shape
b.T
b.T.shape

# Deep Copy
# The copy method makes a complete copy of the array and its data.
d = a.copy()
print(d)
print(d is a)
print(d.base is a)
a = np.arange(12) ** 2
i = np.array([1, 1, 3, 8, 5])
print(a[i])
j = np.array([[3, 4], [9, 7]])
print(a[j])

# Scipy
# There are two (interchangeable) ways to deal with 1-d polynomials in SciPy. The first is to use the poly1d class from Numpy. This class accepts coefficients or polynomial roots to initialize a polynomial. The polynomial object can then be manipulated in algebraic expressions, integrated, differentiated, and evaluated. It even prints like a polynomial:
from numpy import poly1d

p = poly1d([3, 4, 5])
print(p)
print(p * p)
print(p.integ(k=6))
print(p.deriv())
p([4, 5])

# The main feature of the scipy.special package is the definition of numerous special functions of mathematical physics. Available functions include airy, elliptic, bessel, gamma, beta, hypergeometric, parabolic cylinder, mathieu, spheroidal wave, struve, and kelvin. There are also some low-level stats functions that are not intended for general use as an easier interface to these functions is provided by the stats module. Most of these functions can take array arguments and return array results following the same broadcasting rules as other math functions in Numerical Python. Many of these functions also accept complex numbers as input. For a complete list of the available functions with a one-line description type >>> help(special). Each function also has its own documentation accessible using help. If you don’t see a function you need, consider writing it and contributing it to the library. You can write the function in either C, Fortran, or Python. Look in the source code of the library for examples of each of these kinds of functions.
from scipy import special


def drumhead_height(n, k, distance, angle, t):
    kth_zero = special.jn_zeros(n, k)[-1]
    return np.cos(t) * np.cos(n * angle) * special.jn(n, distance * kth_zero)


theta = np.r_[0 : 2 * np.pi : 50j]
radius = np.r_[0:1:50j]
x = np.array([r * np.cos(theta) for r in radius])
y = np.array([r * np.sin(theta) for r in radius])
z = np.array([drumhead_height(1, 1, r, theta, 0.5) for r in radius])
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm

fig = plt.figure()
ax = Axes3D(fig)
ax.plot_surface(x, y, z, rstride=1, cstride=1, cmap=cm.jet)
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_zlabel("Z")
plt.show()


# FIR Filter
# The function firwin designs filters according to the window method. Depending on the provided arguments, the function returns different filter types (e.g. low-pass, band-pass…).
# The example below designs a low-pass and a band-stop filter, respectively.
import numpy as np
import scipy.signal as signal
import matplotlib.pyplot as plt

b1 = signal.firwin(40, 0.5)
b2 = signal.firwin(41, [0.3, 0.8])
w1, h1 = signal.freqz(b1)
w2, h2 = signal.freqz(b2)
plt.title("Digital filter frequency response")
plt.plot(w1, 20 * np.log10(np.abs(h1)), "b")
plt.plot(w2, 20 * np.log10(np.abs(h2)), "r")
plt.ylabel("Amplitude Response (dB)")
plt.xlabel("Frequency (rad/sample)")
plt.grid()
plt.show()
import numpy as np  # linear algebra
import pandas as pd  # data processing, CSV file I/O (e.g. pd.read_csv)
import seaborn as sns
import matplotlib.pyplot as plt
import os

print(os.listdir("../input"))
# read_csv load data
data = pd.read_csv("../input/googleplaystore.csv")
# first 5 rows
data.head()
# last 5 rows
data.tail()
data.sample(5)
data.sample(frac=0.1)
# Can you show your data's columns?
data.columns
# so data's info
data.info()
# Empty fields within the data set
data.isnull().sum()
# It appears that the value of the rating is a little empty. We're gonna have to fill it.
from sklearn.preprocessing import Imputer

im = Imputer(missing_values="NaN", strategy="mean")
data.iloc[:, 2:3] = im.fit_transform(data.iloc[:, 2:3])
# so,We've done adding implants to the empty spaces by taking the mean.
data.isnull().sum()
data.head()

# value_counts(): Frequency counts
# outliers: the value that is considerably higher or lower from rest of the data
# Lets say value at 75% is Q3 and value at 25% is Q1.
# Outlier are smaller than Q1 - 1.5(Q3-Q1) and bigger than Q3 + 1.5(Q3-Q1). (Q3-Q1) = IQR
# We will use describe() method. Describe method includes:
# count: number of entries
# mean: average of entries
# std: standart deviation
# min: minimum entry
# 25%: first quantile
# 50%: median or second quantile
# 75%: third quantile
# max: maximum entry
# What is quantile?
# 1,4,5,6,8,9,11,12,13,14,15,16,17
# The median is the number that is in middle of the sequence. In this case it would be 11.
# The lower quartile is the median in between the smallest number and the median i.e. in between 1 and 11, which is 6.
# The upper quartile, you find the median between the median and the largest number i.e. between 11 and 17, which will be 14 according to the question above.
#
# For example lets look frequency of pokemom types
print(
    data["Category"].value_counts(dropna=False)
)  # if there are nan values that also be counted
# As it can be seen below there are 112 water pokemon or 70 grass pokemon
# Category Analysis
sns.barplot(
    x=data["Category"].value_counts().index, y=data["Category"].value_counts().values
)
plt.xlabel("Category")
plt.ylabel("Count")
plt.title("Category Count Operation")
plt.xticks(rotation=90)
plt.show()
# It seems that the most family and game categories are traded.
fig, ax = plt.subplots()
ax.scatter(
    x=data.groupby("Category")["Rating"].mean()[1:].index,
    y=data.groupby("Category")["Rating"].mean()[1:].values,
)
plt.ylabel("Category", fontsize=13)
plt.xlabel("Rating", fontsize=13)
plt.xticks(rotation=90)
plt.show()
ax = sns.boxplot(x="Rating", y="Size", data=data)
plt.show()
data[data["Reviews"] == max(data.Reviews)]
# most viewed and displayed application
data[data["Reviews"] == min(data.Reviews)]


# least viewed and displayed application
# scaling and cleaning size of installation
def change_size(size):
    if "M" in size:
        x = size[:-1]
        x = float(x) * 1000000
        return x
    elif "k" == size[-1:]:
        x = size[:-1]
        x = float(x) * 1000
        return x
    else:
        return None


data["Size"] = data["Size"].map(change_size)
# filling Size which had NA
data.Size.fillna(method="ffill", inplace=True)
data.head()
filter_data = data[data["Size"] > int(data["Size"].mean())]
filter_data.Reviews = filter_data.Reviews.astype(float)
filter_data[filter_data["Reviews"] > 15].Type.value_counts()
sns.countplot(filter_data[filter_data["Reviews"] > 15].Type)
plt.title("Free vs Paid")
plt.show()
filter_data[(filter_data["Reviews"] > 15) & (filter_data["Type"] == "Paid")]
# The value of the type in the data we filter is analyzed for payment. For this, the total price value will be calculated.
# For this, we need to remove the $ symbol from the charge column. In addition, it is necessary to change the type of milk.
filter_data["Price"] = filter_data.Price.str.replace("$", "")
filter_data.Price = filter_data.Price.astype("float")
# Calculating the total fee
(
    str(
        sum(
            filter_data[
                (filter_data["Reviews"] > 15) & (filter_data["Type"] == "Paid")
            ].Price
        )
    )
    + " $"
)
# filter_data[(filter_data['Reviews']>15)&(filter_data['Type']=='Paid')]
df_filter = filter_data.groupby("Category")["Price"].sum()
# sum and display of paid products for the filtered area in terms of categorical
d1 = pd.DataFrame(df_filter.index, columns=["Category"])
d2 = pd.DataFrame(df_filter.values, columns=["Price"])
df_data_filter = pd.concat([d1, d2], axis=1)
df_data_filter = df_data_filter.sort_values(by="Price", ascending=False)
df_data_filter
cmap = sns.diverging_palette(220, 10, as_cmap=True)
# Draw the heatmap with the mask and correct aspect ratio
sns.heatmap(
    data.corr(),
    cmap=cmap,
    vmax=0.3,
    center=0,
    square=True,
    linewidths=0.5,
    cbar_kws={"shrink": 0.5},
    annot=True,
)
plt.show()
plt.figure(figsize=(5, 5))
plt.pie(
    df_data_filter.Price[0:5],
    labels=df_data_filter.Category[0:5],
    autopct="%1.1f%%",
    shadow=True,
    startangle=90,
)
# ax.axis('Price')  # Equal aspect ratio ensures that pie is drawn as a circle.
# plt.legend()
plt.show()
# sns.relplot(x="Rating",y="Reviews",data=data)
# plt.show()
# sns.lmplot(x="Review", y="Rating", hue="Category",data=data)
# plt.show()
data.dropna(inplace=True)
everyone = []
teen = []
everyone10 = []
mature17 = []
adults18 = []
unrates = []
everyone.append(sum(data[(data["Content Rating"] == "Everyone")].Size))
teen.append(sum(data[(data["Content Rating"] == "Teen")].Size))
mature17.append(sum(data[(data["Content Rating"] == "Mature 17+")].Size))
adults18.append(sum(data[(data["Content Rating"] == "Adults only 18+")].Size))
unrates.append(sum(data[(data["Content Rating"] == "Unrated")].Size))
everyone10.append(sum(data[(data["Content Rating"] == "Everyone 10+")].Size))
# data.groupby('Content Rating')['Size'].mean().values
plt.plot(
    data["Content Rating"].unique(),
    [everyone, teen, mature17, adults18, unrates, everyone10],
)
plt.xlabel("Content Rating")
plt.ylabel("Sum Of Size")
plt.xticks(rotation=90)
plt.show()
data["Content Rating"].unique()
# everyone
data.head()
len(data[data["Category"] == "FAMILY"])
sns.barplot(
    x=data["Category"].unique(), y=data.groupby("Category")["Rating"].mean().values
)
plt.xticks(rotation=90)
plt.xlabel("Category")
plt.ylabel("Mean")
plt.show()
data.head()
data.Price = data.Price.str.replace("$", "")
print(max(data[data["Type"] == "Paid"].Price.values))
print(min(data[data["Type"] == "Paid"].Price.values))
paid_filter = data[data["Type"] == "Paid"]
paid_filter.Price = paid_filter.Price.astype(float)
paid_filter.Reviews = paid_filter.Reviews.astype(float)
paid_filter.groupby("Category")["Reviews"].sum().values
plt.scatter(
    x=paid_filter.Category.unique(),
    y=paid_filter.groupby("Category")["Reviews"].sum().values,
)
plt.xticks(rotation=90)
plt.show()
data.Genres.unique()
countplot = data.Genres.value_counts()
plt.figure(figsize=(10, 10))
sns.barplot(x=countplot.index[:50], y=countplot[:50])
plt.xticks(rotation=90)
plt.title("Count Genres of DataSet")
plt.show()
data[data["Genres"] == "Tools"].Type.value_counts().index
explode = [0, 0.1]
labels = ["Free", "Paid"]
plt.figure(figsize=(5, 5))
plt.pie(
    data[data["Genres"] == "Tools"].Type.value_counts().values,
    labels=data[data["Genres"] == "Tools"].Type.value_counts().index,
    explode=explode,
    autopct="%1.1f%%",
)
plt.title("Genres Tools Type System")
plt.show()
# Show the joint distribution using kernel density estimation
g = sns.jointplot(data.Rating, data.Reviews, kind="kde", height=7, space=0)
plt.show()
data.head()
data.Reviews = data.Reviews.astype(float)
s = paid_filter[paid_filter["Price"] == 0.99].Category.unique()
paid_filter[paid_filter["Price"] == 0.99].groupby("Category")["Rating"].mean()
plt.figure(1, figsize=(15, 7))
plt.subplot(131)
plt.bar(
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .index,
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .values,
)
plt.xticks(rotation=90)
plt.subplot(132)
plt.scatter(
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .index,
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .values,
)
plt.xticks(rotation=90)
plt.subplot(133)
plt.plot(
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .index,
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .values,
)
plt.suptitle("Categorical Plotting")
plt.xticks(rotation=90)
plt.tight_layout()
plt.show()
data.head()
# Plot miles per gallon against horsepower with other semantics
sns.relplot(
    x="Rating",
    y="Reviews",
    hue="Type",
    sizes=(40, 400),
    alpha=0.5,
    palette="muted",
    height=6,
    data=data,
)
plt.show()
g = sns.PairGrid(data, diag_sharey=False)
g.map_lower(sns.kdeplot)
g.map_upper(sns.scatterplot)
g.map_diag(sns.kdeplot, lw=3)
plt.show()
ax = sns.kdeplot(data.Rating, data.Size, cmap="Blues", shade=True)
plt.show()
data.head()
data.Type = [1 if type == "Free" else 0 for type in data.Type]
data.head(2)
data.columns
col = {
    "Content Rating": "ContentRating",
    "Last Updated": "LastUpdated",
    "Current Ver": "CurrentVer",
    "Android Ver": "AndroidVer",
}
data.rename(columns=col, inplace=True)


def showColumns(data):
    for i, col in enumerate(data):
        print("{} . columns {}".format(i, col))


showColumns(data.columns)
data.dtypes
data.Price = data.Price.astype("float")
data.Price = [1 if price > 0 else -1 for price in data.Price]
np.mean(data.Price)
np.eye(5)
np.std(data[:5], axis=1)
np.var(data)
<|endoftext|>
[{"Id": 14071740, "ScriptId": 2821193, "ParentScriptVersionId": NaN, "ScriptLanguageId": 9, "AuthorUserId": 1712516, "CreationDate": "05/12/2019 11:37:31", "VersionNumber": 44.0, "Title": "Python Programming Language For Data Scientists", "EvaluationDate": "05/12/2019", "IsChange": true, "TotalLines": 1676.0, "LinesInsertedFromPrevious": 4.0, "LinesChangedFromPrevious": 0.0, "LinesUnchangedFromPrevious": 1672.0, "LinesInsertedFromFork": NaN, "LinesDeletedFromFork": NaN, "LinesChangedFromFork": NaN, "LinesUnchangedFromFork": NaN, "TotalVotes": 2}]
[{"Id": 9179491, "KernelVersionId": 14071740, "SourceDatasetVersionId": 274957}]
[{"Id": 274957, "DatasetId": 49864, "DatasourceVersionId": 287262, "CreatorUserId": 2115707, "LicenseName": "Unknown", "CreationDate": "02/03/2019 13:55:47", "VersionNumber": 6.0, "Title": "Google Play Store Apps", "Slug": "google-play-store-apps", "Subtitle": "Web scraped data of 10k Play Store apps for analysing the Android market.", "Description": "### Context\n\nWhile many public datasets (on Kaggle and the like) provide Apple App Store data, there are not many counterpart datasets available for Google Play Store apps anywhere on the web. On digging deeper, I found out that iTunes App Store page deploys a nicely indexed appendix-like structure to allow for simple and easy web scraping. On the other hand, Google Play Store uses sophisticated modern-day techniques (like dynamic page load) using JQuery making scraping more challenging.\n\n\n### Content\n\nEach app (row) has values for catergory, rating, size, and more.\n\n\n### Acknowledgements\n\nThis information is scraped from the Google Play Store. This app information would not be available without it.\n\n\n### Inspiration\n\nThe Play Store apps data has enormous potential to drive app-making businesses to success. Actionable insights can be drawn for developers to work on and capture the Android market!", "VersionNotes": "Insert CC license", "TotalCompressedBytes": 165.0, "TotalUncompressedBytes": 2013308.0}]
[{"Id": 49864, "CreatorUserId": 2115707, "OwnerUserId": 2115707.0, "OwnerOrganizationId": NaN, "CurrentDatasetVersionId": 274957.0, "CurrentDatasourceVersionId": 287262.0, "ForumId": 58489, "Type": 2, "CreationDate": "09/04/2018 18:19:51", "LastActivityDate": "09/04/2018", "TotalViews": 1687374, "TotalDownloads": 222127, "TotalVotes": 4501, "TotalKernels": 921}]
# Teaching Python Programming Language For Data Scientists
# Introduction
#
# What is Python Programming Language?
# Why Python Programming Language?
# Required Libraries
#
# Python for Beginners
#
# Python Installation
# Maths Operations
# Strings Operation
# Lists Operation
# if-elif-else
# While Statement
# For Statement
# Range
# Break and Continue
# Introduction to Functions
# Lambda Expressions
# Documentation Strings
# Data Structure
# Dictionaries
# Modules
# Input and Output
# Reading and Writing Files
# Errors and Exceptions
# Classes
#
# Data Preparation and Processing
#
# Pandas
#
# Create Data
# Get Data
# Prepare Data
# Analyze Data
# Presedent Data
#
#
# NumPy
#
# The Basics
# Array Creation
# Printing Arrays
# Basic Operations
# Universal Functions
# Indexing, Slicing and Iterating
# Shape Manipulation
# Deep Copy
#
#
#
# Feature Work
# References
# Conclusion
# if you like it,please UPVOTE
# Last Update : 12.05.2019
# Introduction
# What is Python Programming Language?
# Python is the programming language. Kdnuggets is the most used programming language in data science with 65.5% according to Analytics data. Open source is free and free software. Python is a programming language that supports the object-oriented programming paradigm. Web, mobile and socket programming etc. is a scripting language with many features. However, it can operate independently on every platform. This is an indispensable programming language for data scientists and other programmers thanks to their diversity. In addition, it has a clear and easy-to-read format compared to other programming languages. With its wide library, it helps us in all kinds of software works. All software will be written in this programming language.
# Link: https://www.python.org/downloads/
# Why Python Programming Language?
# It is one of the most used programming languages by data scientists. Because there are developers all around the world. In this way, a wide variety of operations are performed. In addition, very few transactions can be performed with very little code. More than one operation is performed.Python is a powerful programming language that is easy to learn. It has a simple but effective approach to effective high-level data structures and object-oriented programming. Python's exquisite syntax and dynamic writing, combined with its interpreted nature, has made it an ideal language for scripting and rapid application development across many platforms.
# The Python interpreter can be easily expanded with new functions and data types implemented in C or C ++ (or other languages that can be called from C). Python is also available as an extension language for customizable applications.
# Required Libraries
# Numpy
# It is a Python library that allows us to perform multiple operations on arrays, including multidimensional arrays, various derived objects (such as masked arrays and matrices) and lots of mathematical, logical, shape manipulation, sorting, sorting, discrete Fourier. Statistical procedures and simulations can be performed using NumPy. NumPy library with C ++ does not cause any loss of performance.
# Pandas
# Pandas is an open source, BSD licensed library, offering high-performance, easy-to-use data structures and data analysis tools for the Python programming language. It is a library designed to work with data, allowing you to run your non-structural data as in structural databases. Pandas is designed for fast and easy data processing, collection and visualization. Therefore, it is a very preferred library for data scientists. In addition, with its own data-editing functions, you can convert your data into any format. Thanks to Pandas, we make preliminary work for our model and make it ready for the model. As a preliminary study, it is necessary to carry out editing studies by detecting missing, slingshot, incompatible and repetitive data. In this way, you can train your model smoothly and get the results properly. To use the Pandas library, we need to upload it to Anaconda.
# Scikit-Learn
# It includes many machine learning algorithms such as classification, regression, clustering, size reduction, model selection and data removal. It is an open source library with BSD license. Together with the algorithms it contains, it creates a fast and secure coding structure. It includes the functions of expressing frequently used mathematical and physical problems in computer environment. In addition, thanks to its own web site you can look at the working principle of the algorithm you want. However, you can learn the mathematical infrastructure of the algorithm. In this way, according to your own data set you can develop the mathematical model of the algorithm yourself. By applying the model you have made on the data set, results can be obtained. Which of these results is effective in the mathematical model you will make out. In addition, you must install the scikit-learn library to Anaconda to use it.
# Seaborn
# Seaborn is a library in Python for making interesting and informative statistical graphs. Built on Matplotlib, numpy has been tightly integrated with the PyData stack, providing support for statistical templates from scipy, statsmodel, pandas data structures. It is used for graphs depicting general distributions that summarize data as in heat maps. Thanks to the many methods included in the content of high-level analysis can be made. However, the user can perform analyzes in interactive and animation structure. Today's data scientists put the seaborn library in place where the matplotlib library is missing. Thanks to Seaborn, interactive plots can be removed. In this way, we will show the dimensions of inter-feature interaction more clearly. Thanks to the different types of plots within itself, we are able to make analyzes in every format.
# Matplotlib
# Matplotlib graphical drawing package is one of the most important tools of scientific programming. With Matplotlib, a very powerful package, data can be visualized interactively, and high quality printouts are available for publishing. Both two- and three-dimensional graphics can be created. However, it is very simple to use. In addition, there are a lot of information about this package on the internet, but for data scientists can take all kinds of analysis. As a result of your analysis, you can save the resulting graph as output. You can then add it as an addition to the report you made later. It will be used to explain the results obtained from various analyzes and evaluations in the application orally and graphically. So much so that we use the algorithm to show us the most accurate results. In this way, we will guide us for further analysis. In addition, our corporate website helps us for all the plots we will use.
# Scipy
# SciPy refers to several related but distinct entities:The SciPy ecosystem, a collection of open source software for scientific computing in Python.The community of people who use and develop this stack.Several conferences dedicated to scientific computing in Python - SciPy, EuroSciPy and SciPy.in.The SciPy library, one component of the SciPy stack, providing many numerical routines.
# Python for Beginners
# Python Installation
# Link: https://www.youtube.com/watch?v=dX2-V2BocqQ
# There is very good expression in this section.
# Maths Operations
# Python can do a variety of mathematical operations. In this case, we do not need to define any variables. This is described in the examples below for this situation.
# The following mathematical operations are given 3 kinds of examples. In the first example, only data from the variable definition are given. Thus, results were obtained. In the second example, a variable is defined. In this example, the operations were performed. In the last example, a decimal data is obtained by dividing int data with decimal data.**
# Before continue with pandas, we need to learn logic, control flow and filtering.
# Comparison operator: ==, , <=
# Boolean operators: and, or ,not
# Filtering pandas
# Comparison operator
print(3 > 2)
print(5 < 4)
print("Apple" in ["App", "Appl", "Apple"])
print(3 != 2)
# Boolean operators
print(True and False)
print(True or False)
# Example 1
print(2 + 2)
# Example 2
a = (50 - 5 * 6) / 4
print(a)
# Example 3
print(8 / 5.0)
# Example 4
print(10 + 80)
print(10 - (25 - 85) + 45)
print(5 * 6 - (2 - 8))
# Example 5
print(17 // 3.0)  # explicit floor division discards the fractional part
# Example 6
print(18 // 4.0)
# The // operator is also provided for doing floor division no
# matter what the operands are.
# Example 7
print(17 % 3)
# Example 6 Mode operation; between the first number and the second number.
# As a result of this operation, the remaining number is obtained.
# With Python, it is possible to use the ** operator to calculate powers
# Example 8
print(5**2)  # 5 squared
# Example 9
print(2**8)  # 2 to power of 8
# The equal sign (=) is used to assign a value to a variable.
# Example 10
width = 8
height = 10
print(width * height)

# Strings Operation
# Besides numbers, Python can also manipulate strings, which can be expressed in several ways. They can be enclosed in single quotes ('...') or double quotes ("...") with the same result. \ can be used to escape quotes:
s1 = "string"
s2 = "data"
print(s1 + " " + s2)
# Strings can be concatenated (glued together) with the
# + operator, and repeated with *:
index = 3
print("un " * index)
# Two or more string literals
print("Py" "thon")
prefix = "Py"
# prefix='thon'
prefix + "thon"
# Strings can be indexed (subscripted)
word = "Python"
print(word[0])  # character in position 0
print(word[5])  # character in position 5
print(word[-1])  # last character
print(word[-2])  # second-last character
# In addition to indexing, slicing is also supported.
# While indexing is used to obtain individual characters,
# slicing allows you to obtain a substring:
print(word[0:2])  # characters from position 0 (included) to 2 (excluded)
print(word[2:5])  # characters from position 2 (included) to 5 (excluded)
# Note how the start is always included, and the end always
# excluded. This makes sure that s[:i] + s[i:] is always equal to s:
print(word[:2] + word[2:])
print(word[:4] + word[4:])
# Slice indices have useful defaults; an omitted first index defaults
# to zero, an omitted second index defaults to the size of the
# string being sliced.
print(word[:2])  # character from the beginning to position 2 (excluded)
print(word[4:])
print(word[-2:])
print(word[1:3])  # character from the beginning to position 3 (excluded)
print("J" + word[1:])
print(word[:2] + "py")

# Lists Operation
# Python knows a number of compound data types, used to group together other values. The most versatile is the list, which can be written as a list of comma-separated values (items) between square brackets. Lists might contain items of different types, but usually the items all have the same type.
squares = [1, 4, 9, 16, 25]
squares
# Like strings (and all other built-in sequence type),
# lists can be indexed and sliced:
print(squares[0])  # indexing returns the item
print(squares[-1])
print(squares[-3:])  # slicing returns a new list
print(squares[:])  # all
# All slice operations return a new list containing the requested
# elements. This means that the following slice returns a new
# (shallow) copy of the list:
squares + [36, 49, 64, 81, 100]
cubes = [1, 8, 27, 65, 125]
4**3
cubes[3] = 64  # replace the wrong value
cubes
# You can also add new items at the end of the list, by using the
# append() method (we will see more about methods later):
cubes.append(216)  # add the cube of 6
cubes.append(7**3)  # and the cube of 7
cubes
# Assignment to slices is also possible, and this can even change
# the size of the list or clear it entirely:
letters = ["a", "b", "c", "d", "e", "f", "g"]
print(letters)
# replace some values
letters[2:5] = ["C", "D", "E"]
print(letters)
# now remove them
letters[2:5] = []
letters
# clear the list by replacing all the elements with an empty list
letters[:] = []
letters
# The built-in function len() also applies to lists:
letters = ["a", "b", "c", "d"]
len(letters)

# if-elif-else
# Decision making is required when we want to execute a code only if a certain condition is satisfied.
# [i + 1 for i in num1 ]: list of comprehension
# i +1: list comprehension syntax
# for i in num1: for loop syntax
# i: iterator
# num1: iterable object
# The if…elif…else statement is used in Python for decision making.
# Perhaps the most well-known statement type is the if statement. For example:
# We use list comprehension for data analysis often.
# list comprehension: collapse for loops for building lists into a single line
# Ex: num1 = [1,2,3] and we want to make it num2 = [2,3,4]. This can be done with for loop. However it is  unnecessarily long. We can make it one line code that is list comprehension.
# Example of list comprehension
num1 = [1, 2, 3]
num2 = [i + 1 for i in num1]
print(num2)
# Conditionals on iterable
num1 = [5, 10, 15]
num2 = [i**2 if i == 10 else i - 5 if i < 7 else i + 5 for i in num1]
print(num2)
# If the number is positive, we print an appropriate message
num = 3
if num > 0:
    print(num, " is a positive number")
print("This is always printed")
num = -1
if num > 0:
    print(num, " is a positive number")
print("This is also always printed")
# Program checks if the number is positive or negative
# And displays an appropriate message
num = 3
# Try these two variations as well.
# num = -5
# num = 0
if num >= 0:
    print("Positive or Zero")
else:
    print("Negative number")
# In this program,
# we check if the number is positive or
# negative or zero and
# display an appropriate message
num = 3.4
# Try these two variations as well:
# num = 0
# num = -4.5
if num > 0:
    print("Positive number")
elif num == 0:
    print("Zero")
else:
    print("Negative number")
# In this program, we input a number
# check if the number is positive or
# negative or zero and display
# an appropriate message
# This time we use nested if
num = float(input("Enter a number: "))
if num >= 0:
    if num == 0:
        print("Zero")
    else:
        print("Positive number")
else:
    print("Negative number")
var = 100
if var == 100:
    print("Value of expression is 100")
print("Good bye!")

# While
# With the while loop we can execute a set of statements as long as a condition is true.
# Stay in loop if condition( i is not equal 5) is true
i = 0
while i != 5:
    print("i is: ", i)
    i += 1
print(i, " is equal to 5")
# Stay in loop if condition( i is not equal 5) is true
lis = [1, 2, 3, 4, 5]
for i in lis:
    print("i is: ", i)
print("")
# Enumerate index and value of list
# index : value = 0:1, 1:2, 2:3, 3:4, 4:5
for index, value in enumerate(lis):
    print(index, " : ", value)
print("")
# For dictionaries
# We can use for loop to achive key and value of dictionary. We learnt key and value at dictionary part.
dictionary = {"spain": "madrid", "france": "paris"}
for key, value in dictionary.items():
    print(key, " : ", value)
print("")
i = 1
while i < 6:
    print(i)
    i += 1
# With the break statement we can stop the loop even if the while condition is true:
i = 1
while i < 6:
    print(i)
    if i == 3:
        break
    i += 1
# With the continue statement we can stop the current iteration, and continue with the next:
i = 0
while i < 6:
    i += 1
    if i == 3:
        continue
    print(i)

# For Statements
# The for statement in Python differs a bit from what you may be used to in C or Pascal. Rather than always iterating over an arithmetic progression of numbers (like in Pascal), or giving the user the ability to define both the iteration step and halting condition (as C), Python’s for statement iterates over the items of any sequence (a list or a string), in the order that they appear in the sequence. For example (no pun intended):
words = ["cats", "window", "defenstrate"]
for w in words:
    print(w, len(w))

# If you need to modify the sequence you are iterating over while inside the loop (for example to duplicate selected items), it is recommended that you first make a copy. Iterating over a sequence does not implicitly make a copy. The slice notation makes this especially convenient:
for w in words[:]:
    if len(w) > 6:
        words.insert(0, w)
for i, w in enumerate(words):
    print((i + 1), ":", w)
# Loop through the letters in the word "banana":
for x in "banana":
    print(x)

# Range
# If you do need to iterate over a sequence of numbers, the built-in function range() comes in handy. It generates lists containing arithmetic progressions:
# To loop through a set of code a specified number of times, we can use the range() function,
# The range() function returns a sequence of numbers, starting from 0 by default, and increments by 1 (by default), and ends at a specified number.
for i in range(10):
    print(i)

# The given end point is never part of the generated list; range(10) generates a list of 10 values, the legal indices for items of a sequence of length 10. It is possible to let the range start at another number, or to specify a different increment (even negative; sometimes this is called the ‘step’):
for i in range(5, 10):
    print(i)
print("#" * 30)
for i in range(0, 10, 3):  # 0 start 10 finish 3 step
    print(i)
print("#" * 30)
for i in range(-10, -100, -30):  # 0 start 10 finish 3 step
    print(i)
a = ["Mary", "had", "a", "little", "lamb"]
for i in range(len(a)):
    print(i, a[i])
import numpy as np

new_list = np.arange(1, 20)
for i in np.arange(1, 30, 2):
    print(i)
# The else keyword in a for loop specifies a block of code to be executed when the loop is finished:
for x in range(6):
    print(x)
else:
    print("Finish")

# Break and Continue
# The break statement, like in C, breaks out of the innermost enclosing for or while loop.
# Loop statements may have an else clause; it is executed when the loop terminates through exhaustion of the list (with for) or when the condition becomes false (with while), but not when the loop is terminated by a break statement. This is exemplified by the following loop, which searches for prime numbers:
for n in range(2, 10):
    for x in range(2, n):
        if n % x == 0:
            print(n, " equal", x, "*", n / x)
            break
    else:
        # loop fell through without finding a factor
        print(n, " is a prime number")

# (Yes, this is the correct code. Look closely: the else clause belongs to the for loop, not the if statement.)
# When used with a loop, the else clause has more in common with the else clause of a try statement than it does that of if statements: a try statement’s else clause runs when no exception occurs, and a loop’s else clause runs when no break occurs. For more on the try statement and exceptions, see Handling Exceptions.
# The continue statement, also borrowed from C, continues with the next iteration of the loop:
# iteration example
name = "ronaldo"
it = iter(name)
print(next(it))  # print next iteration
print(*it)  # print remaining iteration

#
# ITERATORS
# iterable is an object that can return an iterator
# iterable: an object with an associated iter() method
# example: list, strings and dictionaries
# iterator: produces next value with next() method
#
for num in range(2, 10):
    if num % 2 == 0:
        print("Found an even number", num)
        continue
    print("Found a number", num)


# Introduction to Functions
# We can create a function that writes the Fibonacci series to an arbitrary boundary:
def fib(n):
    # print a fibonacci series up to n
    a, b = 0, 1
    while a < n:
        print(a)
        a, b = b, a + b


fib(2000)


def my_function1(country="Turkey"):
    print("I am from " + country)


my_function1()
my_function1("Canada")


# The keyword def introduces a function definition. It must be followed by the function name and the parenthesized list of formal parameters. The statements that form the body of the function start at the next line, and must be indented.
# The most useful form is to specify a default value for one or more arguments. This creates a function that can be called with fewer arguments than it is defined to allow. For example:


def ask_ok(prompt, retries=4, complaint="Yes or no, please!"):
    while True:
        ok = raw_input(prompt)
        if ok in ("y", "ye", "yes"):
            return True
        if ok in ("n", "no", "nop", "nope"):
            return False
        retries = retries - 1
        if retries < 0:
            raise IOError("refusenik user")
        print(complaint)


# This function can be called in several ways:
# giving only the mandatory argument: ask_ok('Do you really want to quit?')
# giving one of the optional arguments: ask_ok('OK to overwrite the file?', 2)
# or even giving all arguments: ask_ok('OK to overwrite the file?', 2, 'Come on, only yes or no!')
# The default values are evaluated at the point of function definition in the defining scope, so that
i = 5


def f(arg=i):
    print(arg)


i = 6
f()


# Important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. For example, the following function accumulates the arguments passed to it on subsequent calls:
def f(a, L=[]):
    L.append(a)
    return L


print(f(1))
print(f(2))
print(f(3))


# If you don’t want the default to be shared between subsequent calls, you can write the function like this instead:
def f(a, L=None):
    if L is None:
        L = []
    L.append(a)
    return L


f([5, 4, 3])


# Functions can also be called using keyword arguments of the form kwarg=value. For instance, the following function:
def parrot(voltage, state="a stiff", action="voom", type="Norwegian Blue"):
    print(
        "-- This parrot wouldn't",
        action,
    )
    print("if you put", voltage, "volts through it.")
    print("-- Lovely plumage, the", type)
    print("-- It's", state, "!")


# accepts one required argument (voltage) and three optional arguments (state, action, and type). This function can be called in any of the following ways:
# accepts one required argument (voltage) and three optional arguments (state, action, and type). This function can be called in any of the following ways:
parrot(1000)  # 1 positional argument
parrot(voltage=1000)  # 1 keyword argument
parrot(voltage=1000000, action="VOOOOOM")  # 2 keyword arguments
parrot(action="VOOOOOM", voltage=1000000)  # 2 keyword arguments
parrot("a million", "bereft of life", "jump")  # 3 positional arguments
parrot("a thousand", state="pushing up the daisies")  # 1 positional, 1 keyword


# When a final formal parameter of the form **name is present, it receives a dictionary (see Mapping Types — dict) containing all keyword arguments except for those corresponding to a formal parameter. This may be combined with a formal parameter of the form *name (described in the next subsection) which receives a tuple containing the positional arguments beyond the formal parameter list. (*name must occur before **name.) For example, if we define a function like this:
def cheeseshop(kind, *arguments, **keywords):
    print("-- Do you have any", kind, "?")
    print("-- I'm sorry, we're all out of", kind)
    for arg in arguments:
        print(arg)
    print("-" * 40)
    keys = sorted(keywords.keys())
    for kw in keys:
        print(kw, ":", keywords[kw])


"""cheeseshop("Limburger", "It's very runny, sir.",
           "It's really very, VERY runny, sir.",
           shopkeeper='Michael Palin',
           client="John Cleese",
           sketch="Cheese Shop Sketch")"""


# The reverse situation occurs when the arguments are already in a list or tuple but need to be unpacked for a function call requiring separate positional arguments. For instance, the built-in range() function expects separate start and stop arguments. If they are not available separately, write the function call with the *-operator to unpack the arguments out of a list or tuple:
print(list(range(3, 6)))  # normal call with separate arguments
args = [3, 6]
list(range(*args))  # call with arguments unpacked from a list


def tri_recursion(k):
    if k > 0:
        result = k + tri_recursion(k - 1)
        print(result)
    else:
        result = 0
    return result


tri_recursion(6)


# Lambda Expressions
# Small anonymous functions can be created with the lambda keyword. This function returns the sum of its two arguments: lambda a, b: a+b. Lambda functions can be used wherever function objects are required. They are syntactically restricted to a single expression. Semantically, they are just syntactic sugar for a normal function definition. Like nested function definitions, lambda functions can reference variables from the containing scope:
def make_incrementor(n):
    return lambda x: x + n


f = make_incrementor(50)
print(f(2))
print(f(1))


# The above example uses a lambda expression to return a function. Another use is to pass a small function as an argument:
pairs = [(1, "one"), (2, "two"), (3, "three"), (4, "four")]
pairs.sort(key=lambda pair: pair[1])
pairs
# A lambda function that adds 10 to the number passed in as an argument, and print the result:
x = lambda a: a + 10
print(x(5))
x = 5
y = lambda x: x**2
y(10)

# A lambda function that adds 10 to the number passed in as an argument, and print the result:
x = lambda a: a + 120
print(x(15))

# A lambda function that multiplies argument a with argument b and print the result:
x = lambda a, b, c: a + b + c
print(x(5, 6, 2))
number_list = [1, 2, 3]
y = map(lambda x: x**2, number_list)
print(list(y))


# The power of lambda is better shown when you use them as an anonymous function inside another function.
# Say you have a function definition that takes one argument, and that argument will be multiplied with an unknown number:
def myfunc(n):
    return lambda a: a * n


mydoubler = myfunc(2)
print(mydoubler(11))


# Documentation Strings
# There are emerging conventions about the content and formatting of documentation strings.
# The first line should always be a short, concise summary of the object’s purpose. For brevity, it should not explicitly state the object’s name or type, since these are available by other means (except if the name happens to be a verb describing a function’s operation). This line should begin with a capital letter and end with a period.
# If there are more lines in the documentation string, the second line should be blank, visually separating the summary from the rest of the description. The following lines should be one or more paragraphs describing the object’s calling conventions, its side effects, etc.
# Here is an example of a multi-line docstring:
def my_fucntion():
    """Do nothing, but document it
    No,really, it doesn't do anything.
    """
    pass


print(my_fucntion)


# Data Structures
# This chapter describes some things you’ve learned about already in more detail, and adds some new things as well.
a = [66.25, 333, 333, 1, 12, 154, 789]
print(a.count(333), a.count(66.25), a.count("x"))
a.insert(2, -1)
a.append(333)
print(a)
a.index(333)
a.remove(333)
a.reverse()
print(a)
a.sort()
print(a.pop())
a

# The list methods make it very easy to use a list as a stack, where the last element added is the first element retrieved (“last-in, first-out”). To add an item to the top of the stack, use append(). To retrieve an item from the top of the stack, use pop() without an explicit index. For example:
stack = [3, 4, 5]
stack.append(6)
stack.append(7)
print(stack)
print(stack.pop())
print(stack.pop())
print(stack)


def f(x):
    return x % 3 == 0 or x % 5 == 0


f(9)


# map(function, sequence) calls function(item) for each of the sequence’s items and returns a list of the return values. For example, to compute some cubes:
def cube(x):
    return x * x * x


cube(5)


# More than one sequence may be passed; the function must then have as many arguments as there are sequences and is called with the corresponding item from each sequence (or None if some sequence is shorter than another). For example:
seq = range(1, 8)
list(seq)


def add(x, y):
    return x + y


add(5, 7)


# List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.
# For example, assume we want to create a list of squares, like:
squares = []
for x in range(10):
    squares.append(x**2)
squares
for x in range(0, 18, 2):
    print(x)

# We can obtain the same result with:
squares = [i * 2 for i in range(1, 10, 1)]
squares
[(x, y) for x in [1, 2, 3] for y in [3, 1, 4] if x != y]
a = [-1, 1, 66, 45, 87, 65, 12, 65]
del a[0]
print(a)
del a[2:4]
print(a)
del a[:]
print(a)
t = 123, 45, 65, 78, "hi!"
print(t[0])
print(t)
# Tuples may be nested:
u = t, (1, 2, 3, 4, 5, 6)
u

# A special problem is the construction of tuples containing 0 or 1 items: the syntax has some extra quirks to accommodate these. Empty tuples are constructed by an empty pair of parentheses; a tuple with one item is constructed by following a value with a comma (it is not sufficient to enclose a single value in parentheses). Ugly, but effective. For example:
empty = ()
singleton = ("hello",)
len(empty)
len(singleton)
print(singleton)

# Dictionaries
# Another useful data type built into Python is the dictionary (see Mapping Types — dict). Dictionaries are sometimes found in other languages as “associative memories” or “associative arrays”. Unlike sequences, which are indexed by a range of numbers, dictionaries are indexed by keys, which can be any immutable type; strings and numbers can always be keys. Tuples can be used as keys if they contain only strings, numbers, or tuples; if a tuple contains any mutable object either directly or indirectly, it cannot be used as a key. You can’t use lists as keys, since lists can be modified in place using index assignments, slice assignments, or methods like append() and extend().
# create dictionary and look its keys and values
dictionary = {"spain": "madrid", "usa": "vegas"}
print(dictionary.keys())
print(dictionary.values())
# Keys have to be immutable objects like string, boolean, float, integer or tubles
# List is not immutable
# Keys are unique
dictionary["spain"] = "barcelona"  # update existing entry
print(dictionary)
dictionary["france"] = "paris"  # Add new entry
print(dictionary)
del dictionary["spain"]  # remove entry with key 'spain'
print(dictionary)
print("france" in dictionary)  # check include or not
dictionary.clear()  # remove all entries in dict
print(dictionary)
tel = {"jack": 4098, "sape": 4139}
tel["guido"] = 4127
print(tel)
print(tel["jack"])
del tel["sape"]
tel["irv"] = 4127
print(tel)
print(tel.keys())
print(tel.values())
"jack" in tel

# The dict() constructor builds dictionaries directly from sequences of key-value pairs:
dict([("sape", 4139), ("guido", 4127), ("jack", 4098)])

# In addition, dict comprehensions can be used to create dictionaries from arbitrary key and value expressions:
{x: x**2 for x in (2, 4, 6)}
for i, v in enumerate(["tic", "tac", "toe"]):
    print(i, v)


# Modules
# If you quit from the Python interpreter and enter it again, the definitions you have made (functions and variables) are lost. Therefore, if you want to write a somewhat longer program, you are better off using a text editor to prepare the input for the interpreter and running it with that file as input instead. This is known as creating a script. As your program gets longer, you may want to split it into several files for easier maintenance. You may also want to use a handy function that you’ve written in several programs without copying its definition into each program.
# A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module’s name (as a string) is available as the value of the global variable __name__. For instance, use your favorite text editor to create a file called fibo.py in the current directory with the following contents:
# Fibonacci numbers module
def fib(n):  # write Fibonacci series up to n
    a, b = 0, 1
    while b < n:
        print(b),
        a, b = b, a + b


def fib2(n):  # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while b < n:
        result.append(b)
        a, b = b, a + b
    return result


#
# What we need to know about functions:
# docstrings: documentation for functions. Example:
# for f():
# """This is docstring for documentation of function f"""
# tuble: sequence of immutable python objects.
# cant modify values
# tuble uses paranthesis like tuble = (1,2,3)
# unpack tuble into several variables like a,b,c = tuble
#
# How can we learn what is built in scope
import builtins

dir(builtins)


# example of what we learn above
def tuble_ex():
    """return defined t tuble"""
    t = (1, 2, 3)
    return t


a, b, c = tuble_ex()
print(a, b, c)

import sys

"""
The built-in function dir() is used to find out which names a 
module defines. It returns a sorted list of strings:
"""
print(dir(sys))

# Input and Output
# There are several ways to present the output of a program; data can be printed in a human-readable form, or written to a file for future use. This chapter will discuss some of the possibilities.
s = "Hello world"
print(str(s))
print(repr(s))
print(str(1.0 / 7.0))
x = 10 * 3.25
y = 200 * 200
s = "The value of x is " + repr(x) + ", and y is " + repr(y) + "..."
print(s)

# Reading and Writing Files
# open() returns a file object, and is most commonly used with two arguments: open(filename, mode).
# Errors and Exceptions
# Until now error messages haven’t been more than mentioned, but if you have tried out the examples you have probably seen some. There are (at least) two distinguishable kinds of errors: syntax errors and exceptions.
# while True:print('Hello World')
# Syntax errors, also known as parsing errors, are perhaps
# he most common kind of complaint you get while you are still
# earning Python:


# It is possible to write programs that handle selected exceptions. Look at the following example, which asks the user for input until a valid integer has been entered, but allows the user to interrupt the program (using Control-C or whatever the operating system supports); note that a user-generated interruption is signalled by raising the KeyboardInterrupt exception.
# while True:
# try:
# x=int(input('Please enter a number :'))
# break
# except ValueError:
# print('Stop!')
# If an exception has an argument, it is printed as the last part (‘detail’) of the message for unhandled exceptions.
# Exception handlers don’t just handle exceptions if they occur immediately in the try clause, but also if they occur inside functions that are called (even indirectly) in the try clause. For example:
def this_fails():
    x = 1 / 10


try:
    this_fails()
except ZeroDivisonError as detail:
    print("Handling run-time error :", detail)


# Classes
# Compared with other programming languages, Python’s class mechanism adds classes with a minimum of new syntax and semantics. It is a mixture of the class mechanisms found in C++ and Modula-3. Python classes provide all the standard features of Object Oriented Programming: the class inheritance mechanism allows multiple base classes, a derived class can override any methods of its base class or classes, and a method can call the method of a base class with the same name. Objects can contain arbitrary amounts and kinds of data. As is true for modules, classes partake of the dynamic nature of Python: they are created at runtime, and can be modified further after creation.
# Class objects support two kinds of operations: attribute references and instantiation.
# Attribute references use the standard syntax used for all attribute references in Python: obj.name. Valid attribute names are all the names that were in the class’s namespace when the class object was created. So, if the class definition looked like this:
class MyClass:
    """A simple example class"""

    i = 12345

    def f(self):
        return "hello world"


# Class instantiation uses function notation. Just pretend that the class object is a parameterless function that returns a new instance of the class. For example (assuming the above class):
x = MyClass()


# Of course, the __init__() method may have arguments for greater flexibility. In that case, arguments given to the class instantiation operator are passed on to __init__(). For example,
class Complex:
    def __init__(self, realpart, imagpart):
        self.r = realpart
        self.i = imagpart


x = Complex(3.0, -4.5)
print(x.i, x.r)


# Now what can we do with instance objects? The only operations understood by instance objects are attribute references. There are two kinds of valid attribute names, data attributes and methods.
# data attributes correspond to “instance variables” in Smalltalk, and to “data members” in C++. Data attributes need not be declared; like local variables, they spring into existence when they are first assigned to. For example, if x is the instance of MyClass created above, the following piece of code will print the value 16, without leaving a trace:
x.counter = 1
while x.counter < 10:
    x.counter = x.counter * 2
print(x.counter)
del x.counter


# Generally speaking, instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class:
class Dog:
    trick = []
    kind = "canine"

    def __init__(self, name):
        self.name = name

    def add_trick(self, trick):
        self.tricks.append(trick)


d = Dog("Fibo")
e = Dog("Buddy")
print(d.kind)
print(e.kind)
print(d.name)
print(e.name)


class Dog:
    tricks = []  # mistaken use of a class variable

    def __init__(self, name):
        self.name = name

    def add_trick(self, trick):
        self.tricks.append(trick)


d = Dog("Fido")
e = Dog("Buddy")
d.add_trick("roll over")
e.add_trick("play dead")
d.tricks  # unexpectedly shared by all dogs
["roll over", "play dead"]


# Data Preparation and Processing
# Pandas
# import pandas as pd
# Create Data
# The data set will consist of 5 baby names and the number of births recorded for that year (1880).
import pandas as pd  # import library as pd

names = ["Bob", "Jessica", "Mary", "Jhon", "Mel"]
births = [968, 45, 756, 12, 65]
# To merge these two lists together we will use the zip function.
BabyDataSet = list(zip(names, births))
print(BabyDataSet)

# We are basically done creating the data set. We now will use the pandas library to export this data set into a csv file.
# df will be a DataFrame object. You can think of this object holding the contents of the BabyDataSet in a format similar to a sql table or an excel spreadsheet. Lets take a look below at the contents inside df.
df = pd.DataFrame(data=BabyDataSet, columns=["Names", "Births"])
df

# Export the dataframe to a csv file. We can name the file births1880.csv. The function to_csv will be used to export the file. The file will be saved in the same location of the notebook unless specified otherwise.
# df.to_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv',index=False,header=False)
# Merge the names and the births data set using the zip function.
BabyDataSet = list(zip(names, births))
BabyDataSet[:10]
# Lets add a column
df["Year"] = 2019
# using loc accessor
df.loc[1:5, ["Names"]]
# Slicing and indexing series
df.loc[1:10, "Names":]  # 10 and "Defense" are inclusive
# replaces the deprecated ix function
# df.ix[5:,'col']
df.loc[df.index[1:], "Names"]
# replaces the deprecated ix function
# df.ix[:3,['col', 'test']]
df.loc[df.index[:3], ["Names", "Year"]]

# Get Data
# To pull in the csv file, we will use the pandas function read_csv. Let us take a look at this function and what inputs it takes.
# df=pd.read_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv')
df

# This brings us to the first problem of the exercise. The read_csv function treated the first record in the csv file as the header names. This is obviously not correct since the text file did not provide us with header names.
# To correct this we will pass the header parameter to the read_csv function and set it to None (means null in python).
##df=pd.read_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv',header=None)
print(df)
# df=pd.read_csv('C:\\Users\\aAa\\Desktop\\Kaggle-Python-Learn\\biths.csv',names=['Name','Birth'])
print(df)
df.info()

# Info says:
# There are 999 records in the data set
# There is a column named Mary with 999 values
# There is a column named 968 with 999 values
# Out of the two columns, one is numeric, the other is non numericli>
# To actually see the contents of the dataframe we can use the head() function which by default will return the first five records. You can also pass in a number n to return the top n records of the dataframe.
df.head(4)
# Now lets take a look at the last five records of the dataframe
df.tail(4)
df.sample()

# Prepare Data
# The data we have consists of baby names and the number of births in the year 1880. We already know that we have 5 records and none of the records are missing (non-null values).
# Realize that aside from the check we did on the "Names" column, briefly looking at the data inside the dataframe should be as far as we need to go at this stage of the game. As we continue in the data analysis life cycle we will have plenty of opportunities to find any issues with the data set.
# Check data type of the columns
df.dtypes
# Check data type of births column
df.Births.dtype
# We can use the unique property of the dataframe to find all the unique records of the "Names" column.
df["Names"].unique()
# If you actually want to print the unique values:
for x in df["Names"].unique():
    print(x)
print(df["Names"].describe())

# Since we have multiple values per baby name, we need to aggregate this data so we only have a baby name appear once. This means the 1,000 rows will need to become 5. We can accomplish this by using the groupby function.
# clean name column convert to upper
df["Names"] = df.Names.apply(lambda x: x.upper())
df
# clean name column convert to upper
df["Names"] = df.Names.apply(lambda x: x.lower())
df
data_filter = data.copy()
data_filter.Genres.apply(lambda x: x.strip("&")).sample(5)
name = df.groupby("Names")
df1 = name.sum()
df1
df

# Analyze Data
# To find the most popular name or the baby name with the higest birth rate, we can do one of the following.
# Sort the dataframe and select the top row
# Use the max() attribute to find the maximum value
#
sorted = df.sort_values(["Births"], ascending=False)
sorted.head(1)
df["Births"].max()

# Present Data
# Here we can plot the Births column and label the graph to show the end user the highest point on the graph. In conjunction with the table, the end user has a clear picture that Mel is the most popular baby name in the data set.
# plot() is a convinient attribute where pandas lets you painlessly plot the data in your dataframe. We learned how to find the maximum value of the Births column in the previous section. Now to find the actual baby name of the 973 value looks a bit tricky, so lets go over it.s
# NumPy
# The Basic
# NumPy’s main object is the homogeneous multidimensional array. It is a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers. In NumPy dimensions are called axes.
import numpy as np

a = np.arange(15).reshape(3, 5)
a
a.shape
a.ndim
a.dtype.name
a.itemsize

# Array Creation
# There are several ways to create arrays.
# For example, you can create an array from a regular Python list or tuple using the array function. The type of the resulting array is deduced from the type of the elements in the sequences.
a = np.array([2, 3, 4])
a
a.dtype
b = np.array([1.2, 3.5, 5.1])
b.dtype

# A frequent error consists in calling array with multiple numeric arguments, rather than providing a single list of numbers as an argument.
# a=np.array(1,2,3,4) #that's not true
a = np.array([1, 2, 3, 4])  # that's true
# array transforms sequences of sequences into two-dimensional
# arrays, sequences of sequences of sequences into three-dimensional arrays, and so on.
b = np.array([(1.5, 2, 3), (4, 5, 6)])
b

# The function zeros creates an array full of zeros, the function ones creates an array full of ones, and the function empty creates an array whose initial content is random and depends on the state of the memory. By default, the dtype of the created array is float64.
np.zeros((3, 4))
np.eye(4, 4)
eyes = np.eye(5, 5)
np.diagonal(eyes)
array = np.array([[2, 3, 4], [6, 7, 8], [10, 11, 12]])
array.T
np.ones((2, 3, 4), dtype=np.int16)
np.empty((2, 3))

# To create sequences of numbers, NumPy provides a function analogous to range that returns arrays instead of lists.
np.arange(10, 30, 5)
np.arange(-1, 1, 0.1)
np.arange(0, 2, 0.3)

# When arange is used with floating point arguments, it is generally not possible to predict the number of elements obtained, due to the finite floating point precision. For this reason, it is usually better to use the function linspace that receives as an argument the number of elements that we want, instead of the step:
np.linspace(0, 2, 9)
np.linspace(0, 2, 10)
from numpy import pi

x = np.linspace(0, 2 * pi, 100)
f = np.sin(x)

# Printing Arrays
# When you print an array, NumPy displays it in a similar way to nested lists, but with the following layout:
# the last axis is printed from left to right,
# the second-to-last is printed from top to bottom,
# the rest are also printed from top to bottom, with each slice separated from the next by an empty line.
#
a = np.arange(6)  # 1d array
print(a)
b = np.arange(12).reshape(4, 3)  # 2d array
print(b)
c = np.arange(24).reshape(2, 3, 4)  # 3d array
print(c)
print(np.arange(10000))
print(np.arange(10000).reshape(100, 100))

# Basic Operations
# Arithmetic operators on arrays apply elementwise. A new array is created and filled with the result.
a = np.array([20, 30, 40, 50])
b = np.arange(4)
print(b)
c = a - b
print(c)
print(b**2)
print(10 * np.sin(a))
print(a < 35)

# Universal Functions
# NumPy provides familiar mathematical functions such as sin, cos, and exp. In NumPy, these are called “universal functions”(ufunc). Within NumPy, these functions operate elementwise on an array, producing an array as output.
B = np.arange(3)
print(B)
np.exp(B)
np.sqrt(B)
C = np.array([2.0, -1.0, 4.0])
np.add(B, C)
np.abs(-100)

# Indexing, Slicing and Iterating
# One-dimensional arrays can be indexed, sliced and iterated over, much like lists and other Python sequences.
a = np.arange(10) ** 3
print(a)
print(a[2])
print(a[2:5])
a[:6:2] = -1100
print(a)

# Shape Manipulation
a = np.floor(10 * np.random.random((3, 4)))
print(a)
a.shape
print(a.shape[0])

# The shape of an array can be changed with various commands. Note that the following three commands all return a modified array, but do not change the original array:
a.ravel()  # returns the array,flattend
a.shape
b.T
b.T.shape

# Deep Copy
# The copy method makes a complete copy of the array and its data.
d = a.copy()
print(d)
print(d is a)
print(d.base is a)
a = np.arange(12) ** 2
i = np.array([1, 1, 3, 8, 5])
print(a[i])
j = np.array([[3, 4], [9, 7]])
print(a[j])

# Scipy
# There are two (interchangeable) ways to deal with 1-d polynomials in SciPy. The first is to use the poly1d class from Numpy. This class accepts coefficients or polynomial roots to initialize a polynomial. The polynomial object can then be manipulated in algebraic expressions, integrated, differentiated, and evaluated. It even prints like a polynomial:
from numpy import poly1d

p = poly1d([3, 4, 5])
print(p)
print(p * p)
print(p.integ(k=6))
print(p.deriv())
p([4, 5])

# The main feature of the scipy.special package is the definition of numerous special functions of mathematical physics. Available functions include airy, elliptic, bessel, gamma, beta, hypergeometric, parabolic cylinder, mathieu, spheroidal wave, struve, and kelvin. There are also some low-level stats functions that are not intended for general use as an easier interface to these functions is provided by the stats module. Most of these functions can take array arguments and return array results following the same broadcasting rules as other math functions in Numerical Python. Many of these functions also accept complex numbers as input. For a complete list of the available functions with a one-line description type >>> help(special). Each function also has its own documentation accessible using help. If you don’t see a function you need, consider writing it and contributing it to the library. You can write the function in either C, Fortran, or Python. Look in the source code of the library for examples of each of these kinds of functions.
from scipy import special


def drumhead_height(n, k, distance, angle, t):
    kth_zero = special.jn_zeros(n, k)[-1]
    return np.cos(t) * np.cos(n * angle) * special.jn(n, distance * kth_zero)


theta = np.r_[0 : 2 * np.pi : 50j]
radius = np.r_[0:1:50j]
x = np.array([r * np.cos(theta) for r in radius])
y = np.array([r * np.sin(theta) for r in radius])
z = np.array([drumhead_height(1, 1, r, theta, 0.5) for r in radius])
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm

fig = plt.figure()
ax = Axes3D(fig)
ax.plot_surface(x, y, z, rstride=1, cstride=1, cmap=cm.jet)
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_zlabel("Z")
plt.show()


# FIR Filter
# The function firwin designs filters according to the window method. Depending on the provided arguments, the function returns different filter types (e.g. low-pass, band-pass…).
# The example below designs a low-pass and a band-stop filter, respectively.
import numpy as np
import scipy.signal as signal
import matplotlib.pyplot as plt

b1 = signal.firwin(40, 0.5)
b2 = signal.firwin(41, [0.3, 0.8])
w1, h1 = signal.freqz(b1)
w2, h2 = signal.freqz(b2)
plt.title("Digital filter frequency response")
plt.plot(w1, 20 * np.log10(np.abs(h1)), "b")
plt.plot(w2, 20 * np.log10(np.abs(h2)), "r")
plt.ylabel("Amplitude Response (dB)")
plt.xlabel("Frequency (rad/sample)")
plt.grid()
plt.show()
import numpy as np  # linear algebra
import pandas as pd  # data processing, CSV file I/O (e.g. pd.read_csv)
import seaborn as sns
import matplotlib.pyplot as plt
import os

print(os.listdir("../input"))
# read_csv load data
data = pd.read_csv("../input/googleplaystore.csv")
# first 5 rows
data.head()
# last 5 rows
data.tail()
data.sample(5)
data.sample(frac=0.1)
# Can you show your data's columns?
data.columns
# so data's info
data.info()
# Empty fields within the data set
data.isnull().sum()
# It appears that the value of the rating is a little empty. We're gonna have to fill it.
from sklearn.preprocessing import Imputer

im = Imputer(missing_values="NaN", strategy="mean")
data.iloc[:, 2:3] = im.fit_transform(data.iloc[:, 2:3])
# so,We've done adding implants to the empty spaces by taking the mean.
data.isnull().sum()
data.head()

# value_counts(): Frequency counts
# outliers: the value that is considerably higher or lower from rest of the data
# Lets say value at 75% is Q3 and value at 25% is Q1.
# Outlier are smaller than Q1 - 1.5(Q3-Q1) and bigger than Q3 + 1.5(Q3-Q1). (Q3-Q1) = IQR
# We will use describe() method. Describe method includes:
# count: number of entries
# mean: average of entries
# std: standart deviation
# min: minimum entry
# 25%: first quantile
# 50%: median or second quantile
# 75%: third quantile
# max: maximum entry
# What is quantile?
# 1,4,5,6,8,9,11,12,13,14,15,16,17
# The median is the number that is in middle of the sequence. In this case it would be 11.
# The lower quartile is the median in between the smallest number and the median i.e. in between 1 and 11, which is 6.
# The upper quartile, you find the median between the median and the largest number i.e. between 11 and 17, which will be 14 according to the question above.
#
# For example lets look frequency of pokemom types
print(
    data["Category"].value_counts(dropna=False)
)  # if there are nan values that also be counted
# As it can be seen below there are 112 water pokemon or 70 grass pokemon
# Category Analysis
sns.barplot(
    x=data["Category"].value_counts().index, y=data["Category"].value_counts().values
)
plt.xlabel("Category")
plt.ylabel("Count")
plt.title("Category Count Operation")
plt.xticks(rotation=90)
plt.show()
# It seems that the most family and game categories are traded.
fig, ax = plt.subplots()
ax.scatter(
    x=data.groupby("Category")["Rating"].mean()[1:].index,
    y=data.groupby("Category")["Rating"].mean()[1:].values,
)
plt.ylabel("Category", fontsize=13)
plt.xlabel("Rating", fontsize=13)
plt.xticks(rotation=90)
plt.show()
ax = sns.boxplot(x="Rating", y="Size", data=data)
plt.show()
data[data["Reviews"] == max(data.Reviews)]
# most viewed and displayed application
data[data["Reviews"] == min(data.Reviews)]


# least viewed and displayed application
# scaling and cleaning size of installation
def change_size(size):
    if "M" in size:
        x = size[:-1]
        x = float(x) * 1000000
        return x
    elif "k" == size[-1:]:
        x = size[:-1]
        x = float(x) * 1000
        return x
    else:
        return None


data["Size"] = data["Size"].map(change_size)
# filling Size which had NA
data.Size.fillna(method="ffill", inplace=True)
data.head()
filter_data = data[data["Size"] > int(data["Size"].mean())]
filter_data.Reviews = filter_data.Reviews.astype(float)
filter_data[filter_data["Reviews"] > 15].Type.value_counts()
sns.countplot(filter_data[filter_data["Reviews"] > 15].Type)
plt.title("Free vs Paid")
plt.show()
filter_data[(filter_data["Reviews"] > 15) & (filter_data["Type"] == "Paid")]
# The value of the type in the data we filter is analyzed for payment. For this, the total price value will be calculated.
# For this, we need to remove the $ symbol from the charge column. In addition, it is necessary to change the type of milk.
filter_data["Price"] = filter_data.Price.str.replace("$", "")
filter_data.Price = filter_data.Price.astype("float")
# Calculating the total fee
(
    str(
        sum(
            filter_data[
                (filter_data["Reviews"] > 15) & (filter_data["Type"] == "Paid")
            ].Price
        )
    )
    + " $"
)
# filter_data[(filter_data['Reviews']>15)&(filter_data['Type']=='Paid')]
df_filter = filter_data.groupby("Category")["Price"].sum()
# sum and display of paid products for the filtered area in terms of categorical
d1 = pd.DataFrame(df_filter.index, columns=["Category"])
d2 = pd.DataFrame(df_filter.values, columns=["Price"])
df_data_filter = pd.concat([d1, d2], axis=1)
df_data_filter = df_data_filter.sort_values(by="Price", ascending=False)
df_data_filter
cmap = sns.diverging_palette(220, 10, as_cmap=True)
# Draw the heatmap with the mask and correct aspect ratio
sns.heatmap(
    data.corr(),
    cmap=cmap,
    vmax=0.3,
    center=0,
    square=True,
    linewidths=0.5,
    cbar_kws={"shrink": 0.5},
    annot=True,
)
plt.show()
plt.figure(figsize=(5, 5))
plt.pie(
    df_data_filter.Price[0:5],
    labels=df_data_filter.Category[0:5],
    autopct="%1.1f%%",
    shadow=True,
    startangle=90,
)
# ax.axis('Price')  # Equal aspect ratio ensures that pie is drawn as a circle.
# plt.legend()
plt.show()
# sns.relplot(x="Rating",y="Reviews",data=data)
# plt.show()
# sns.lmplot(x="Review", y="Rating", hue="Category",data=data)
# plt.show()
data.dropna(inplace=True)
everyone = []
teen = []
everyone10 = []
mature17 = []
adults18 = []
unrates = []
everyone.append(sum(data[(data["Content Rating"] == "Everyone")].Size))
teen.append(sum(data[(data["Content Rating"] == "Teen")].Size))
mature17.append(sum(data[(data["Content Rating"] == "Mature 17+")].Size))
adults18.append(sum(data[(data["Content Rating"] == "Adults only 18+")].Size))
unrates.append(sum(data[(data["Content Rating"] == "Unrated")].Size))
everyone10.append(sum(data[(data["Content Rating"] == "Everyone 10+")].Size))
# data.groupby('Content Rating')['Size'].mean().values
plt.plot(
    data["Content Rating"].unique(),
    [everyone, teen, mature17, adults18, unrates, everyone10],
)
plt.xlabel("Content Rating")
plt.ylabel("Sum Of Size")
plt.xticks(rotation=90)
plt.show()
data["Content Rating"].unique()
# everyone
data.head()
len(data[data["Category"] == "FAMILY"])
sns.barplot(
    x=data["Category"].unique(), y=data.groupby("Category")["Rating"].mean().values
)
plt.xticks(rotation=90)
plt.xlabel("Category")
plt.ylabel("Mean")
plt.show()
data.head()
data.Price = data.Price.str.replace("$", "")
print(max(data[data["Type"] == "Paid"].Price.values))
print(min(data[data["Type"] == "Paid"].Price.values))
paid_filter = data[data["Type"] == "Paid"]
paid_filter.Price = paid_filter.Price.astype(float)
paid_filter.Reviews = paid_filter.Reviews.astype(float)
paid_filter.groupby("Category")["Reviews"].sum().values
plt.scatter(
    x=paid_filter.Category.unique(),
    y=paid_filter.groupby("Category")["Reviews"].sum().values,
)
plt.xticks(rotation=90)
plt.show()
data.Genres.unique()
countplot = data.Genres.value_counts()
plt.figure(figsize=(10, 10))
sns.barplot(x=countplot.index[:50], y=countplot[:50])
plt.xticks(rotation=90)
plt.title("Count Genres of DataSet")
plt.show()
data[data["Genres"] == "Tools"].Type.value_counts().index
explode = [0, 0.1]
labels = ["Free", "Paid"]
plt.figure(figsize=(5, 5))
plt.pie(
    data[data["Genres"] == "Tools"].Type.value_counts().values,
    labels=data[data["Genres"] == "Tools"].Type.value_counts().index,
    explode=explode,
    autopct="%1.1f%%",
)
plt.title("Genres Tools Type System")
plt.show()
# Show the joint distribution using kernel density estimation
g = sns.jointplot(data.Rating, data.Reviews, kind="kde", height=7, space=0)
plt.show()
data.head()
data.Reviews = data.Reviews.astype(float)
s = paid_filter[paid_filter["Price"] == 0.99].Category.unique()
paid_filter[paid_filter["Price"] == 0.99].groupby("Category")["Rating"].mean()
plt.figure(1, figsize=(15, 7))
plt.subplot(131)
plt.bar(
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .index,
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .values,
)
plt.xticks(rotation=90)
plt.subplot(132)
plt.scatter(
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .index,
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .values,
)
plt.xticks(rotation=90)
plt.subplot(133)
plt.plot(
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .index,
    paid_filter[paid_filter["Price"] == 0.99]
    .groupby("Category")["Rating"]
    .mean()
    .values,
)
plt.suptitle("Categorical Plotting")
plt.xticks(rotation=90)
plt.tight_layout()
plt.show()
data.head()
# Plot miles per gallon against horsepower with other semantics
sns.relplot(
    x="Rating",
    y="Reviews",
    hue="Type",
    sizes=(40, 400),
    alpha=0.5,
    palette="muted",
    height=6,
    data=data,
)
plt.show()
g = sns.PairGrid(data, diag_sharey=False)
g.map_lower(sns.kdeplot)
g.map_upper(sns.scatterplot)
g.map_diag(sns.kdeplot, lw=3)
plt.show()
ax = sns.kdeplot(data.Rating, data.Size, cmap="Blues", shade=True)
plt.show()
data.head()
data.Type = [1 if type == "Free" else 0 for type in data.Type]
data.head(2)
data.columns
col = {
    "Content Rating": "ContentRating",
    "Last Updated": "LastUpdated",
    "Current Ver": "CurrentVer",
    "Android Ver": "AndroidVer",
}
data.rename(columns=col, inplace=True)


def showColumns(data):
    for i, col in enumerate(data):
        print("{} . columns {}".format(i, col))


showColumns(data.columns)
data.dtypes
data.Price = data.Price.astype("float")
data.Price = [1 if price > 0 else -1 for price in data.Price]
np.mean(data.Price)
np.eye(5)
np.std(data[:5], axis=1)
np.var(data)
