__04.data_analyzing_crawling_youtube_ranking (유튜브 1편)

• 유튜브 2편 전처리/시각화: https://lime-jelly.tistory.com/42

__05.data.analyzing_youtube_data_visualizing (유튜브 2편)

4.1. 크롤링 전 사전 준비

• 이해가 안된다면 음악차트 크롤링을 참고하세요

4.1.1. HTML 다운하기

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from bs4 import BeautifulSoup

import pandas as pd
import numpy as np
from pandas import Series, DataFrame

ser = Service('../chromedriver/chromedriver.exe')
driver = webdriver.Chrome(service = ser)

url= 'https://youtube-rank.com/board/bbs/board.php?bo_table=youtube'
driver.get(url)

html_youtube = driver.page_source

soup = BeautifulSoup(html_youtube, 'html.parser')
soup

4.1.2 크롤링 하기

you = soup.select('tr.aos-init')
print(len(you))
# 100
channel = you[0]
channel

# ['장르','채널명', '구독자수','View 수','동영상 수']
data_list = []
rank = 1
channel = you[0]

for channel in you:
    genre = channel.select('h1 > p')[0].text.strip()
    name = channel.select('h1 > a')[0].text.strip()
    subscribers = channel.select('td.subscriber_cnt')[0].text.strip()
    views = channel.select('td.view_cnt')[0].text.strip()
    videos = channel.select('td.video_cnt')[0].text.strip()
    print(genre, name, subscribers, views, videos, sep='|')
    my_list = ['Youtube', rank, genre, name, subscribers, views, videos]
    data_list.append(my_list)
    rank += 1
'''
[음악/댄스/가수]|BLACKPINK|7130만|220억1380만|395개
[음악/댄스/가수]|HYBE LABELS|6360만|209억9925만|786개
[음악/댄스/가수]|BANGTANTV|6290만|146억5398만|1,665개
[음악/댄스/가수]|SMTOWN|2970만|234억2536만|3,840개
[키즈/어린이]|Boram Tube Vlog [보람튜브 브이로그]|2650만|110억5288만|223개
[음악/댄스/가수]|1MILLION Dance Studio|2470만|70억0346만|3,513개
[음악/댄스/가수]|1theK (원더케이)|2380만|215억4029만|15,831개
[음악/댄스/가수]|JYP Entertainment|2340만|153억4262만|1,435개
[음악/댄스/가수]|Mnet K-POP|1920만|142억1154만|27,004개
[TV/방송]|KBS WORLD TV|1740만|130억6447만|54,745개
[음악/댄스/가수]|JFlaMusic|1740만|35억2897만|282개
[음식/요리/레시피]|Jane ASMR 제인|1580만|56억9319만|1,337개
[음악/댄스/가수]|officialpsy|1540만|89억1343만|91개
[음악/댄스/가수]|BIGBANG|1400만|68억8354만|773개
[음악/댄스/가수]|TWICE|1290만|28억6640만|727개
[음식/요리/레시피]|Hongyu ASMR 홍유|1120만|32억8265만|432개
[키즈/어린이]|Nao FunFun|1070만|459만|37개
[음악/댄스/가수]|Stone Music Entertainment|1020만|83억7984만|8,442개
[키즈/어린이]|DuDuPopTOY|1020만|45억7419만|1,958개
[키즈/어린이]|핑크퐁 (인기 동요・동화)|1010만|61억1515만|2,441개
[키즈/어린이]|Toymong tv 토이몽TV|974만|40억1866만|1,953개
[TV/방송]|MBCkpop|971만|77억5495만|42,103개
[키즈/어린이]|Larva TUBA|957만|41억8739만|2,974개
[키즈/어린이]|서은이야기[SeoeunStory]|943만|40억7111만|1,036개
[음악/댄스/가수]|Mnet TV|875만|103억4348만|19,490개
[음식/요리/레시피]|[햄지]Hamzy|868만|26억1349만|380개
[음악/댄스/가수]|EXO|861만|5억5296만|240개
[뉴스/정치/사회]|MBCentertainment|854만|150억8052만|88,851개
[음악/댄스/가수]|TOMORROW X TOGETHER OFFICIAL|852만|6억2898만|941개
[음악/댄스/가수]|iKON|845만|21억8121만|475개
[음악/댄스/가수]|Stray Kids|838만|14억0622만|537개
[TV/방송]|M2|791만|60억5227만|15,098개
[해외]|Serie A|775만|26억4493만|21,489개
[키즈/어린이]|MariAndKids|755만|23억2910만|810개
[음식/요리/레시피]|쏘영 Ssoyoung|736만|7억5240만|530개
[음악/댄스/가수]|SEVENTEEN|735만|24억6276만|986개
[키즈/어린이]|SonicToy소닉토이|728만|30억2714만|163개
[음악/댄스/가수]|이지금 [IU Official]|726만|12억2765만|139개
[음식/요리/레시피]|문복희 Eat with Boki|717만|17억0724만|513개
[뉴스/정치/사회]|JTBC Entertainment|708만|102억5022만|67,302개
[음악/댄스/가수]|Sungha Jung|680만|18억9597만|1,301개
[TV/방송]|KBS Kpop|664만|52억0200만|31,067개
[미분류]|Pixar|655만|32억4455만|695개
[음악/댄스/가수]|ITZY|654만|12억5214만|521개
[음악/댄스/가수]|ALL THE K-POP|653만|40억3666만|21,260개
[음악/댄스/가수]|YG ENTERTAINMENT|653만|16억6432만|389개
[음악/댄스/가수]|MAMAMOO|644만|18억2727만|633개
[TV/방송]|스브스케이팝 / SBS KPOP|628만|43억5120만|27,883개
[음악/댄스/가수]|GOT7|614만|7억8583만|579개
[TV/방송]|SBS Entertainment|606만|71억9905만|219,201개
[TV/방송]|tvN drama|599만|61억4655만|38,861개
[회사/오피셜]|Samsung|595만|13억1879만|1,634개
[패션/미용]|PONY Syndrome|592만|3억4649만|198개
[키즈/어린이]|[장난감티비]TOYTV|586만|28억7435만|1,545개
[TV/방송]|SBS Drama|585만|61억7267만|341,975개
[키즈/어린이]|미니특공대TV|583만|32억8217만|1,208개
[음식/요리/레시피]|야미보이 Yummyboy|580만|21억3466만|911개
[키즈/어린이]|스위트티비 SweetTV|562만|27억3075만|513개
[음악/댄스/가수]|2NE1|539만|20억0710만|321개
[음식/요리/레시피]|백종원의 요리비책 Paik's Cuisine|526만|5억9360만|373개
[음식/요리/레시피]|tzuyang쯔양|526만|8억5221만|308개
[키즈/어린이]|콩순이 Kongsuni|520만|46억4845만|1,296개
[BJ/인물/연예인]|[Awesome Haeun]어썸하은|518만|8억2337만|521개
[음악/댄스/가수]|starshipTV|508만|29억4523만|1,999개
[키즈/어린이]|베이비버스 -인기 동요・동화|492만|33억4406만|1,357개
[키즈/어린이]|ToyMart TV|491만|21억4933만|651개
[음악/댄스/가수]|TREASURE (트레저)|490만|9억7809만|316개
[음악/댄스/가수]|BT21|489만|4억0378만|220개
[키즈/어린이]|TOY GUMA|489만|16억1396만|306개
[뉴스/정치/사회]|SBS NOW / SBS 공식 채널|479만|48억0676만|20,621개
[음악/댄스/가수]|NCT|477만|3억8207만|293개
[키즈/어린이]|With Kids Playground [위드키즈 놀이터]|474만|17억8165만|416개
[음악/댄스/가수]|Red Velvet|468만|7억4214만|154개
[음식/요리/레시피]|푸메Fume|464만|9억6115만|513개
[TV/방송]|KBS Entertain|463만|75억0905만|103,370개
[음식/요리/레시피]|떵개떵|460만|26억2627만|4,465개
[음악/댄스/가수]|ASTRO 아스트로|457만|4억5619만|475개
[TV/방송]|MBCdrama|454만|51억2523만|55,247개
[BJ/인물/연예인]|영국남자 Korean Englishman|454만|14억8114만|447개
[키즈/어린이]|뽀로로(Pororo)|450만|61억3635만|3,734개
[음식/요리/레시피]|하루한끼 one meal a day|440만|4억3182만|176개
[음악/댄스/가수]|NCT 127|436만|5억1086만|297개
[TV/방송]|JTBC Drama|435만|44억2272만|23,899개
[TV/방송]|채널 NCT DAILY|434만|7억7380만|467개
[음악/댄스/가수]|NCT DREAM|432만|5억0057만|245개
[애완/반려동물]|SBS TV동물농장x애니멀봐|431만|39억4209만|3,249개
[음식/요리/레시피]|Cooking tree 쿠킹트리|431만|4억0526만|1,192개
[키즈/어린이]|슈슈토이 Shushu ToysReview|429만|16억1604만|656개
[게임]|EA SPORTS FIFA|426만|8억0475만|796개
[음식/요리/레시피]|까니짱 [ G-NI ]|425만|11억3391만|554개
[음악/댄스/가수]|Raon Lee|424만|9억7013만|301개
[키즈/어린이]|로미유 스토리[Romiyu Story]|419만|11억0449만|434개
[음식/요리/레시피]|[Dorothy]도로시|416만|10억0227만|907개
[취미/라이프]|JaeYeol ASMR 재열|400만|11억1348만|1,037개
[키즈/어린이]|CoCosToy 꼬꼬스토이|396만|22억5115만|664개
[BJ/인물/연예인]|허팝Heopop|385만|33억0441만|1,912개
[TV/방송]|워크맨-Workman|381만|7억7678만|163개
[음악/댄스/가수]|WINNER|381만|12억2780만|461개
[애완/반려동물]|크림히어로즈|377만|14억3043만|1,027개
[키즈/어린이]|Lime Tube[라임튜브]|375만|26억6897만|1,833개
'''

4.1.3. excel 저장

pd.data = pd.DataFrame(data_list, columns = ['서비스','랭킹','장르', '채널이름','구독수', 'view', '동영상 수'])
pd.data.to_excel('./Chapter_3_1/files/youtube_rank_class.xlsx', na_rep='NAN', index = False, sheet_name = 'youtube_channel_ranking')

유튜브 자료 비주얼라이징 (visualizing) : https://lime-jelly.tistory.com/42