scrapy???????????????????????????(python??????)

Posted 2021-02-11

?????????settings   init   div   start   ?????????   col   tin   pytho   mod   

???.?????????scrapy?

scrapy?????????????????????????????????,?????????????????????????????????????????????,????????????,????????????,?????????????????????????????????????????????????????????(?????????????????????,??????,?????????,????????????)???????????????????????????????????????,??????????????????,????????????????????????????????????,???????????????????????????.

???.??????

??????Linux???

      pip3 install scrapy
 
??????Windows???

      a. pip3 install wheel

      b. ??????twisted http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

      c. ??????????????????????????? pip3 install Twisted?17.1.0?cp35?cp35m?win_amd64.whl

      d. pip3 install pywin32

      e. pip3 install scrapy

 ???.????????????

1.????????????: scrapy startproject ????????????

????????????:

project_name/
   scrapy.cfg???
   project_name/
       __init__.py
       items.py
       pipelines.py
       settings.py
       spiders/
           __init__.py

scrapy.cfg    ????????????????????????.(????????????????????????????????????settings.py?????????)
items.py      ????????????????????????,????????????????????? ,???django???Model
pipelines     ?????????????????????
settings.py   ????????????,???:???????????????,?????????,???????????????
spider        ????????????, ???:????????????,????????????????????????

 

 ??????...