Fix this code

A lot of real data is messy. It might contain zeros, empty values, or NaN's (not-a-number) that we need to watch out for when performing calculations!

Alright, back to our CEOs. Check out the '% Change' column in compensation. It shows the percentage increase in the CEO's pay from the previous year. For CEOs with no previous year on record, it instead says '(No previous year)'. The values in this column are strings, not numbers, so like the 'Total Pay' column, it's not usable without a bit of extra work.

Given your current pay and the percentage increase from the previous year, you can compute your previous year's pay. This year's pay and last year's pay are related by the formula:

this year's pay=last year's pay⋅(1+percent increase100)
For example, if you made $600 last year and your pay increased by 50%, then

this year's pay=600⋅(1+50100)=600⋅(1.5)=900
Since we know how this year's pay and last year's pay are mathematically related, we can solve this equation for last year's pay to give a formula for the previous year's pay if we know the current year's pay and the percent increase from the previous year to the current year. Doing some algebra, we find:

last year's pay=this year's pay(1+percent increase100)
Question 4.6. Create a new DataFrame called with_previous_compensation. It should be a copy of compensation, but with the '(No previous year)' CEOs filtered out, and with an extra column called 'Previous_Total_Pay'. That column should have each CEO's pay in 2014.

Hints:

This question takes several steps, but each one involves something you've already learned how to do. Take it one step at a time, using as many lines as you need. You can print out your results after each step to make sure you're on the right track.
You'll need to define a function to convert percentage strings, like '-15%', to numbers. You can do that just above your other code.
If you see an error saying unsupported operand type(s) for +: 'int' and 'Series', try switching the order of your operation. Instead of writing something like 1 + series, write series + 1.
Note: Don't use the ~ symbol anywhere inside your code, or the Gradescope autograder may not be able to grade your work.

# You can write code to perform intermediate steps here!
with_previous_compensation = ...
with_previous_compensation
Ellipsis
grader.check("q4_6")

q4_6 results:

q4_6 - 1 result:
    Trying:
        "(No previous year)" not in with_previous_compensation.get("% Change")
    Expecting:
        True
    **********************************************************************
    Line 1, in q4_6 0
    Failed example:
        "(No previous year)" not in with_previous_compensation.get("% Change")
    Exception raised:
        Traceback (most recent call last):
          File "/opt/conda/lib/python3.9/doctest.py", line 1336, in __run
            exec(compile(example.source, filename, "single",
          File "", line 1, in 
            "(No previous year)" not in with_previous_compensation.get("% Change")
        AttributeError: 'ellipsis' object has no attribute 'get'

q4_6 - 2 result:
    Trying:
        t = with_previous_compensation.sort_values(by="Previous_Total_Pay", ascending=False)
    Expecting nothing
    **********************************************************************
    Line 1, in q4_6 1
    Failed example:
        t = with_previous_compensation.sort_values(by="Previous_Total_Pay", ascending=False)
    Exception raised:
        Traceback (most recent call last):
          File "/opt/conda/lib/python3.9/doctest.py", line 1336, in __run
            exec(compile(example.source, filename, "single",
          File "", line 1, in 
            t = with_previous_compensation.sort_values(by="Previous_Total_Pay", ascending=False)
        AttributeError: 'ellipsis' object has no attribute 'sort_values'
    Trying:
        value = t.get("Previous_Total_Pay").values[0]
    Expecting nothing
    **********************************************************************
    Line 2, in q4_6 1
    Failed example:
        value = t.get("Previous_Total_Pay").values[0]
    Exception raised:
        Traceback (most recent call last):
          File "/opt/conda/lib/python3.9/doctest.py", line 1336, in __run
            exec(compile(example.source, filename, "single",
          File "", line 1, in 
            value = t.get("Previous_Total_Pay").values[0]
        NameError: name 't' is not defined
    Trying:
        np.isclose(value, 67700000.0)
    Expecting:
        True
    **********************************************************************
    Line 3, in q4_6 1
    Failed example:
        np.isclose(value, 67700000.0)
    Exception raised:
        Traceback (most recent call last):
          File "/opt/conda/lib/python3.9/doctest.py", line 1336, in __run
            exec(compile(example.source, filename, "single",
          File "", line 1, in 
            np.isclose(value, 67700000.0)
        NameError: name 'value' is not defined

q4_6 - 3 result:
    Trying:
        with_previous_compensation.shape[0] == 81
    Expecting:
        True
    **********************************************************************
    Line 1, in q4_6 2
    Failed example:
        with_previous_compensation.shape[0] == 81
    Exception raised:
        Traceback (most recent call last):
          File "/opt/conda/lib/python3.9/doctest.py", line 1336, in __run
            exec(compile(example.source, filename, "single",
          File "", line 1, in 
            with_previous_compensation.shape[0] == 81
        AttributeError: 'ellipsis' object has no attribute 'shape'
Question 4.7. What was the average pay of these CEOs in 2014?

average_pay_2014 = ...
average_pay_2014
Ellipsis
grader.check("q4_7")

q4_7 results:

q4_7 - 1 result:
    Trying:
        np.isclose(average_pay_2014, 11649176.11)
    Expecting:
        True
    **********************************************************************
    Line 1, in q4_7 0
    Failed example:
        np.isclose(average_pay_2014, 11649176.11)
    Exception raised:
        Traceback (most recent call last):
          File "/opt/conda/lib/python3.9/doctest.py", line 1336, in __run
            exec(compile(example.source, filename, "single",
          File "", line 1, in 
            np.isclose(average_pay_2014, 11649176.11)
          File "<__array_function__ internals>", line 5, in isclose
          File "/opt/conda/lib/python3.9/site-packages/numpy/core/numeric.py", line 2355, in isclose
            xfin = isfinite(x)
        TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
5. Distribution of Wealth ??
Earlier, we computed the average pay among the CEOs in our 102-CEO dataset. The average doesn't tell us everything about the amounts CEOs are paid, though. Maybe just a few CEOs make the bulk of the money, even among these 102.

We can use a density histogram to display more information about a set of numbers. Here's an example of the code that we use to create histograms.

Question 5.1. Make a density histogram of the pay of the CEOs in compensation.

# Place your code here
compensation["Total_Pay_Dollars"].plot(kind="hist", x = "CEO Total Pay in Dollars", y = "Density", bins=20, density=True, alpha=0.75)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
/tmp/ipykernel_176/3975829467.py in <module>
      1 # Place your code here
----> 2 compensation["Total_Pay_Dollars"].plot(kind="hist", x = "CEO Total Pay in Dollars", y = "Density", bins=20, density=True, alpha=0.75)

/opt/conda/lib/python3.9/site-packages/babypandas/bpd.py in __getitem__(self, key)
    103             key = key.to_ser()
    104         if not com.is_bool_indexer(key):
--> 105             raise IndexError('BabyPandas only accepts Boolean objects '
    106                              'when indexing against the data frame; '
    107                              'please use .get to get columns, and '

IndexError: BabyPandas only accepts Boolean objects when indexing against the data frame; please use .get to get columns, and .loc or .iloc for more complex cases.

Question 5.2. Looking at the histogram, about how many CEOs made more than $30 million? Make a good guess. (Answer the question by filling in your answer manually. You'll have to do a bit of arithmetic; feel free to use Python as a calculator.)

4
num_ceos_more_than_30_million = 4
grader.check("q5_2")

q5_2 passed!
Question 5.3. Answer the same question with code. That is, find the number of CEOs with a total pay of over $30 million using code, and assign the result to the name num_ceos_more_than_30_million_code.

num_ceos_more_than_30_million_code = ...
num_ceos_more_than_30_million_code = (compensation["Total_Pay_Dollars"] > 30000000).sum() num_ceos_more_than_30_million_code
num_ceos_more_than_30_million_code
  File "/tmp/ipykernel_176/3858755500.py", line 1
    num_ceos_more_than_30_million_code = (compensation["Total_Pay_Dollars"] > 30000000).sum() num_ceos_more_than_30_million_code
                                                                                              ^
SyntaxError: invalid syntax


grader.check("q5_3")

q5_3 results:

q5_3 - 1 result:
    Trying:
        num_ceos_more_than_30_million_code == 5
    Expecting:
        True
    **********************************************************************
    Line 1, in q5_3 0
    Failed example:
        num_ceos_more_than_30_million_code == 5
    Expected:
        True
    Got:
        False
Run the next cell if you want to see how far off you were.

percent_diff = abs(num_ceos_more_than_30_million - num_ceos_more_than_30_million_code) / num_ceos_more_than_30_million_code
print(f"Your guess was only {percent_diff * 100}% off!")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_176/2150670378.py in <module>
----> 1 percent_diff = abs(num_ceos_more_than_30_million - num_ceos_more_than_30_million_code) / num_ceos_more_than_30_million_code
      2 print(f"Your guess was only {percent_diff * 100}% off!")

TypeError: unsupported operand type(s) for -: 'ellipsis' and 'ellipsis'