import pandas as pd

# Get data from URL (what we did in workshop)
# df = pd.read_csv('http://bit.ly/airbnbcsv')

# Get data from this folder (you need the CSV in the same folder as this notebook.)
df = pd.read_csv('airbnb.csv')

df

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 48895 entries, 0 to 48894
Data columns (total 16 columns):
id                                48895 non-null int64
name                              48879 non-null object
host_id                           48895 non-null int64
host_name                         48874 non-null object
neighbourhood_group               48895 non-null object
neighbourhood                     48895 non-null object
latitude                          48895 non-null float64
longitude                         48895 non-null float64
room_type                         48895 non-null object
price                             48895 non-null int64
minimum_nights                    48895 non-null int64
number_of_reviews                 48895 non-null int64
last_review                       38843 non-null object
reviews_per_month                 38843 non-null float64
calculated_host_listings_count    48895 non-null int64
availability_365                  48895 non-null int64
dtypes: float64(3), int64(7), object(6)
memory usage: 6.0+ MB

df['price'].describe()

count    48884.000000
mean       152.755053
std        240.170260
min         10.000000
25%         69.000000
50%        106.000000
75%        175.000000
max      10000.000000
Name: price, dtype: float64

Remove rooms that cost zero per night

price_is_zero = df['price'] == 0

# The ~ or "bitwise" operator flips the series of bolleans (true/false values)
df = df[~price_is_zero]

df['price'].describe()

count    48884.000000
mean       152.755053
std        240.170260
min         10.000000
25%         69.000000
50%        106.000000
75%        175.000000
max      10000.000000
Name: price, dtype: float64

How many entries in each neighborhood?

df['neighbourhood'].value_counts()

Williamsburg                  3919
Bedford-Stuyvesant            3710
Harlem                        2658
Bushwick                      2462
Upper West Side               1971
Hell's Kitchen                1958
East Village                  1853
Upper East Side               1798
Crown Heights                 1564
Midtown                       1545
East Harlem                   1117
Greenpoint                    1114
Chelsea                       1113
Lower East Side                911
Astoria                        900
Washington Heights             899
West Village                   768
Financial District             744
Flatbush                       621
Clinton Hill                   572
Long Island City               537
Prospect-Lefferts Gardens      535
Park Slope                     506
East Flatbush                  500
Fort Greene                    489
Murray Hill                    484
Kips Bay                       470
Flushing                       426
Ridgewood                      423
Greenwich Village              392
                              ... 
Emerson Hill                     5
New Dorp Beach                   5
New Brighton                     5
Oakwood                          5
Prince's Bay                     4
Olinville                        4
Castleton Corners                4
Arden Heights                    4
Holliswood                       4
Mill Basin                       4
Todt Hill                        4
Spuyten Duyvil                   4
Neponsit                         3
Huguenot                         3
Eltingville                      3
Graniteville                     3
Breezy Point                     3
West Farms                       2
Lighthouse Hill                  2
Silver Lake                      2
Co-op City                       2
Howland Hook                     2
Westerleigh                      2
Bay Terrace, Staten Island       2
Woodrow                          1
Richmondtown                     1
Willowbrook                      1
Fort Wadsworth                   1
Rossville                        1
New Dorp                         1
Name: neighbourhood, Length: 221, dtype: int64

Is Morningside Heights in the data set?

# Set function gets unique values in a list or series

'Morningside Heights' in set(df['neighbourhood']) 

True

How many neightborhoods in data set?

len(set(df['neighbourhood']))

221

Let's find out the most pricey neighborhoods

df.groupby('neighbourhood')['price'].mean().sort_values(ascending=False)

neighbourhood
Fort Wadsworth        800.000000
Woodrow               700.000000
Tribeca               490.638418
Sea Gate              487.857143
Riverdale             442.090909
Prince's Bay          409.500000
Battery Park City     367.557143
Flatiron District     341.925000
Randall Manor         336.000000
NoHo                  295.717949
SoHo                  287.103352
Midtown               282.719094
Neponsit              274.666667
West Village          267.682292
Greenwich Village     263.405612
Chelsea               249.738544
Willowbrook           249.000000
Theater District      248.013889
Nolita                230.138340
Financial District    225.490591
Gramercy              222.754438
Little Italy          222.066116
Murray Hill           221.415289
Breezy Point          213.333333
Cobble Hill           211.929293
Upper West Side       210.918316
Brooklyn Heights      209.064935
Hell's Kitchen        204.794178
Kips Bay              202.408511
DUMBO                 196.305556
                         ...    
Fieldston              75.083333
Rossville              75.000000
Concourse Village      73.781250
Westerleigh            71.500000
Highbridge             71.111111
Silver Lake            70.000000
University Heights     69.571429
Fordham                69.444444
Morris Park            69.333333
Schuylerville          69.230769
Parkchester            69.076923
Graniteville           68.666667
Emerson Hill           68.200000
Arden Heights          67.250000
Woodhaven              67.170455
Olinville              64.000000
Borough Park           63.066176
Castle Hill            63.000000
Woodlawn               60.090909
Corona                 59.171875
Mount Eden             58.500000
Concord                58.192308
Grant City             57.666667
New Dorp Beach         57.400000
Bronxdale              57.105263
New Dorp               57.000000
Soundview              53.466667
Tremont                51.545455
Hunts Point            50.500000
Bull's Head            47.333333
Name: price, Length: 221, dtype: float64

What's the average price in my old neighborhood (Sunnyside)?

sunnyside = df[df['neighbourhood'] == 'Sunnyside']

sunnyside['price'].mean()

84.86501377410468

sunnyside['price'].describe()

count    363.000000
mean      84.865014
std       52.227837
min       12.000000
25%       50.000000
50%       75.000000
75%      100.000000
max      600.000000
Name: price, dtype: float64

From FAQ: How do we get a dataframe with only certain columsn?

df[['price', 'neighbourhood']]