使用 ghost.py 抓取页面时出错

Posted

技术标签:

【中文标题】使用 ghost.py 抓取页面时出错【英文标题】:Error using ghost.py to scrape a page 【发布时间】:2013-10-04 21:32:47 【问题描述】:

我正在尝试使用 ghost.py 使用可在我的本地计算机上运行的脚本来抓取网站:

import pdb
from ghost import Ghost
from urlparse import urlparse
import urllib2

...

def scrape(self, request):
    pdb.set_trace()
    page_url = request.DATA.get('url')
    page = urllib2.urlopen(page_url).read()
    original_url = urlparse(page_url)

    print original_url.scheme
    print original_url.netloc

    ghost = Ghost(wait_timeout = 90, display = True)
    page, resources = ghost.open(page_url)

    #use ghost.py to evaluate the javascript on the page
    result, resources = ghost.evaluate("""
                                var a = document.getElementsByTagName('img');
                                var SrcAlt = [];
                                for (var i = 0; i<a.length; i++)
                                    var src = a[i].getAttribute('src');
                                    var alt = a[i].getAttribute('alt');
                                    SrcAlt.push("src": src, "alt": alt);
                                
                                SrcAlt;
                            """)
    #counter = 0
    recs=[]
    for i in result:
        src=None
        alt=None
        try:
            # Gets the image src
            src = unicode(i[PyQt4.QtCore.QString(u'src')])
            try:
                # Gets the image alt tags
                alt = unicode(i[PyQt4.QtCore.QString(u'alt')])
            except:
                pass
        except:
            pass

但是,当我将它添加到我的 Web 应用程序时,我得到了下面这个非常长的错误,并且 Python 崩溃了。我已经阅读了错误的每一行,但无法弄清楚问题所在。任何帮助都会让我知道让它发挥作用的第一个方向!谢谢!!!

/Users/fitvalet/wgwt/web/wgwt/views.py(171)scrape()
    -> page_url = request.DATA.get('url')
    (Pdb) c
    http
    www1.macys.com
    2013-09-28 14:38:07.441 python[2475:1e0b] *** Assertion failure in +[NSUndoManager _endTopLevelGroupings], /SourceCache/Foundation/Foundation-945.18/Misc.subproj/NSUndoManager.m:328
    2013-09-28 14:38:07.441 python[2475:1e0b] +[NSUndoManager(NSInternal) _endTopLevelGroupings] is only safe to invoke on the main thread.
    2013-09-28 14:38:07.442 python[2475:1e0b] (
        0   CoreFoundation                      0x00007fff8f846b06 __exceptionPreprocess + 198
        1   libobjc.A.dylib                     0x00007fff8f56a3f0 objc_exception_throw + 43
        2   CoreFoundation                      0x00007fff8f846948 +[NSException raise:format:arguments:] + 104
        3   Foundation                          0x00007fff8fcfa4c2 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 189
        4   Foundation                          0x00007fff8fd60807 +[NSUndoManager(NSPrivate) _endTopLevelGroupings] + 156
        5   AppKit                              0x00007fff8b4ae24d -[NSApplication run] + 687
        6   QtGui                               0x00000001086da51d _ZN26QEventDispatcherMacPrivate22ensureNSAppInitializedEv + 99
        7   QtGui                               0x00000001086d9e14 _ZN19QEventDispatcherMac13processEventsE6QFlagsIN10QEventLoop17ProcessEventsFlagEE + 480
        8   QtCore                              0x00000001092918b1 _ZN16QCoreApplication13processEventsE6QFlagsIN10QEventLoop17ProcessEventsFlagEE + 81
        9   QtCore.so                           0x00000001098b27f9 meth_QCoreApplication_processEvents + 164
        10  Python                              0x0000000103e9f5a9 PyEval_EvalFrameEx + 9244
        11  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        12  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        13  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        14  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        15  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        16  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        17  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        18  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        19  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        20  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        21  Python                              0x0000000103e956c6 PyObject_Call + 97
        22  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        23  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        24  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        25  Python                              0x0000000103e956c6 PyObject_Call + 97
        26  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        27  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        28  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        29  Python                              0x0000000103e956c6 PyObject_Call + 97
        30  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        31  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        32  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        33  Python                              0x0000000103e956c6 PyObject_Call + 97
        34  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        35  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        36  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        37  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        38  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        39  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        40  Python                              0x0000000103e956c6 PyObject_Call + 97
        41  Python                              0x0000000103eb29bf PyMethod_New + 1627
        42  Python                              0x0000000103e956c6 PyObject_Call + 97
        43  Python                              0x0000000103f231a2 _PyObject_SlotCompare + 11443
        44  Python                              0x0000000103e956c6 PyObject_Call + 97
        45  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        46  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        47  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        48  Python                              0x0000000103e956c6 PyObject_Call + 97
        49  Python                              0x0000000103eb29bf PyMethod_New + 1627
        50  Python                              0x0000000103e956c6 PyObject_Call + 97
        51  Python                              0x0000000103f231a2 _PyObject_SlotCompare + 11443
        52  Python                              0x0000000103e956c6 PyObject_Call + 97
        53  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        54  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        55  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        56  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        57  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        58  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        59  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        60  Python                              0x0000000103e956c6 PyObject_Call + 97
        61  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        62  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        63  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        64  Python                              0x0000000103e956c6 PyObject_Call + 97
        65  Python                              0x0000000103eb29bf PyMethod_New + 1627
        66  Python                              0x0000000103e956c6 PyObject_Call + 97
        67  Python                              0x0000000103f23a33 _PyObject_SlotCompare + 13636
        68  Python                              0x0000000103f1d4d9 _PyType_Lookup + 1493
        69  Python                              0x0000000103e956c6 PyObject_Call + 97
        70  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        71  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        72  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        73  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        74  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        75  Python                              0x0000000103e956c6 PyObject_Call + 97
        76  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        77  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        78  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        79  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        80  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        81  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        82  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        83  Python                              0x0000000103e956c6 PyObject_Call + 97
        84  Python                              0x0000000103eb29bf PyMethod_New + 1627
        85  Python                              0x0000000103e956c6 PyObject_Call + 97
        86  Python                              0x0000000103ea3018 PyEval_CallObjectWithKeywords + 177
        87  Python                              0x0000000103eff7f6 initthread + 1224
        88  libsystem_c.dylib                   0x00007fff8b1a7772 _pthread_start + 327
        89  libsystem_c.dylib                   0x00007fff8b1941a1 thread_start + 13
    )
    2013-09-28 14:38:07.444 python[2475:1e0b] *** Assertion failure in +[NSUndoManager _endTopLevelGroupings], /SourceCache/Foundation/Foundation-945.18/Misc.subproj/NSUndoManager.m:328
    2013-09-28 14:38:07.444 python[2475:1e0b] An uncaught exception was raised
    2013-09-28 14:38:07.444 python[2475:1e0b] +[NSUndoManager(NSInternal) _endTopLevelGroupings] is only safe to invoke on the main thread.
    2013-09-28 14:38:07.445 python[2475:1e0b] (
        0   CoreFoundation                      0x00007fff8f846b06 __exceptionPreprocess + 198
        1   libobjc.A.dylib                     0x00007fff8f56a3f0 objc_exception_throw + 43
        2   CoreFoundation                      0x00007fff8f846948 +[NSException raise:format:arguments:] + 104
        3   Foundation                          0x00007fff8fcfa4c2 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 189
        4   Foundation                          0x00007fff8fd60807 +[NSUndoManager(NSPrivate) _endTopLevelGroupings] + 156
        5   AppKit                              0x00007fff8b4ae30a -[NSApplication run] + 876
        6   QtGui                               0x00000001086da51d _ZN26QEventDispatcherMacPrivate22ensureNSAppInitializedEv + 99
        7   QtGui                               0x00000001086d9e14 _ZN19QEventDispatcherMac13processEventsE6QFlagsIN10QEventLoop17ProcessEventsFlagEE + 480
        8   QtCore                              0x00000001092918b1 _ZN16QCoreApplication13processEventsE6QFlagsIN10QEventLoop17ProcessEventsFlagEE + 81
        9   QtCore.so                           0x00000001098b27f9 meth_QCoreApplication_processEvents + 164
        10  Python                              0x0000000103e9f5a9 PyEval_EvalFrameEx + 9244
        11  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        12  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        13  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        14  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        15  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        16  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        17  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        18  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        19  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        20  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        21  Python                              0x0000000103e956c6 PyObject_Call + 97
        22  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        23  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        24  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        25  Python                              0x0000000103e956c6 PyObject_Call + 97
        26  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        27  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        28  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        29  Python                              0x0000000103e956c6 PyObject_Call + 97
        30  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        31  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        32  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        33  Python                              0x0000000103e956c6 PyObject_Call + 97
        34  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        35  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        36  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        37  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        38  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        39  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        40  Python                              0x0000000103e956c6 PyObject_Call + 97
        41  Python                              0x0000000103eb29bf PyMethod_New + 1627
        42  Python                              0x0000000103e956c6 PyObject_Call + 97
        43  Python                              0x0000000103f231a2 _PyObject_SlotCompare + 11443
        44  Python                              0x0000000103e956c6 PyObject_Call + 97
        45  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        46  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        47  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        48  Python                              0x0000000103e956c6 PyObject_Call + 97
        49  Python                              0x0000000103eb29bf PyMethod_New + 1627
        50  Python                              0x0000000103e956c6 PyObject_Call + 97
        51  Python                              0x0000000103f231a2 _PyObject_SlotCompare + 11443
        52  Python                              0x0000000103e956c6 PyObject_Call + 97
        53  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        54  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        55  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        56  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        57  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        58  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        59  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        60  Python                              0x0000000103e956c6 PyObject_Call + 97
        61  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        62  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        63  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        64  Python                              0x0000000103e956c6 PyObject_Call + 97
        65  Python                              0x0000000103eb29bf PyMethod_New + 1627
        66  Python                              0x0000000103e956c6 PyObject_Call + 97
        67  Python                              0x0000000103f23a33 _PyObject_SlotCompare + 13636
        68  Python                              0x0000000103f1d4d9 _PyType_Lookup + 1493
        69  Python                              0x0000000103e956c6 PyObject_Call + 97
        70  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        71  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        72  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        73  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        74  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        75  Python                              0x0000000103e956c6 PyObject_Call + 97
        76  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        77  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        78  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        79  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        80  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        81  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        82  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        83  Python                              0x0000000103e956c6 PyObject_Call + 97
        84  Python                              0x0000000103eb29bf PyMethod_New + 1627
        85  Python                              0x0000000103e956c6 PyObject_Call + 97
        86  Python                              0x0000000103ea3018 PyEval_CallObjectWithKeywords + 177
        87  Python                              0x0000000103eff7f6 initthread + 1224
        88  libsystem_c.dylib                   0x00007fff8b1a7772 _pthread_start + 327
        89  libsystem_c.dylib                   0x00007fff8b1941a1 thread_start + 13
    )
    2013-09-28 14:38:07.448 python[2475:1e0b] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: '+[NSUndoManager(NSInternal) _endTopLevelGroupings] is only safe to invoke on the main thread.'
    *** First throw call stack:
    (
        0   CoreFoundation                      0x00007fff8f846b06 __exceptionPreprocess + 198
        1   libobjc.A.dylib                     0x00007fff8f56a3f0 objc_exception_throw + 43
        2   CoreFoundation                      0x00007fff8f846948 +[NSException raise:format:arguments:] + 104
        3   Foundation                          0x00007fff8fcfa4c2 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 189
        4   Foundation                          0x00007fff8fd60807 +[NSUndoManager(NSPrivate) _endTopLevelGroupings] + 156
        5   AppKit                              0x00007fff8b4ae30a -[NSApplication run] + 876
        6   QtGui                               0x00000001086da51d _ZN26QEventDispatcherMacPrivate22ensureNSAppInitializedEv + 99
        7   QtGui                               0x00000001086d9e14 _ZN19QEventDispatcherMac13processEventsE6QFlagsIN10QEventLoop17ProcessEventsFlagEE + 480
        8   QtCore                              0x00000001092918b1 _ZN16QCoreApplication13processEventsE6QFlagsIN10QEventLoop17ProcessEventsFlagEE + 81
        9   QtCore.so                           0x00000001098b27f9 meth_QCoreApplication_processEvents + 164
        10  Python                              0x0000000103e9f5a9 PyEval_EvalFrameEx + 9244
        11  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        12  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        13  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        14  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        15  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        16  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        17  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        18  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        19  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        20  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        21  Python                              0x0000000103e956c6 PyObject_Call + 97
        22  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        23  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        24  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        25  Python                              0x0000000103e956c6 PyObject_Call + 97
        26  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        27  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        28  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        29  Python                              0x0000000103e956c6 PyObject_Call + 97
        30  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        31  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        32  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        33  Python                              0x0000000103e956c6 PyObject_Call + 97
        34  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        35  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        36  Python                              0x0000000103ea38df _PyEval_SliceIndex + 989
        37  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        38  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        39  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        40  Python                              0x0000000103e956c6 PyObject_Call + 97
        41  Python                              0x0000000103eb29bf PyMethod_New + 1627
        42  Python                              0x0000000103e956c6 PyObject_Call + 97
        43  Python                              0x0000000103f231a2 _PyObject_SlotCompare + 11443
        44  Python                              0x0000000103e956c6 PyObject_Call + 97
        45  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        46  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        47  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        48  Python                              0x0000000103e956c6 PyObject_Call + 97
        49  Python                              0x0000000103eb29bf PyMethod_New + 1627
        50  Python                              0x0000000103e956c6 PyObject_Call + 97
        51  Python                              0x0000000103f231a2 _PyObject_SlotCompare + 11443
        52  Python                              0x0000000103e956c6 PyObject_Call + 97
        53  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        54  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        55  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        56  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        57  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        58  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        59  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        60  Python                              0x0000000103e956c6 PyObject_Call + 97
        61  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        62  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        63  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        64  Python                              0x0000000103e956c6 PyObject_Call + 97
        65  Python                              0x0000000103eb29bf PyMethod_New + 1627
        66  Python                              0x0000000103e956c6 PyObject_Call + 97
        67  Python                              0x0000000103f23a33 _PyObject_SlotCompare + 13636
        68  Python                              0x0000000103f1d4d9 _PyType_Lookup + 1493
        69  Python                              0x0000000103e956c6 PyObject_Call + 97
        70  Python                              0x0000000103e9f78d PyEval_EvalFrameEx + 9728
        71  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        72  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        73  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        74  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        75  Python                              0x0000000103e956c6 PyObject_Call + 97
        76  Python                              0x0000000103e9df82 PyEval_EvalFrameEx + 3573
        77  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        78  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        79  Python                              0x0000000103ea3869 _PyEval_SliceIndex + 871
        80  Python                              0x0000000103e9f63a PyEval_EvalFrameEx + 9389
        81  Python                              0x0000000103e9d147 PyEval_EvalCodeEx + 1934
        82  Python                              0x0000000103ed6d7a PyFunction_SetClosure + 872
        83  Python                              0x0000000103e956c6 PyObject_Call + 97
        84  Python                              0x0000000103eb29bf PyMethod_New + 1627
        85  Python                              0x0000000103e956c6 PyObject_Call + 97
        86  Python                              0x0000000103ea3018 PyEval_CallObjectWithKeywords + 177
        87  Python                              0x0000000103eff7f6 initthread + 1224
        88  libsystem_c.dylib                   0x00007fff8b1a7772 _pthread_start + 327
        89  libsystem_c.dylib                   0x00007fff8b1941a1 thread_start + 13
    )
    libc++abi.dylib: terminate called throwing an exception
    bin/run-server-dev.sh: line 4:  2473 Abort trap: 6           python web/manage.py runserver "0.0.0.0:$PORT"
    (env)FitValet-MacBook-Pro:wgwt fitvalet$ 

【问题讨论】:

【参考方案1】:

回溯指向wgwt/web/notification_central/views.pyline 14,表示它正在尝试访问AnonymousUser 对象(不存在)的userprofile 属性。

一种解决方法是确保您已登录...(这样request.user 就是一个auth.User 对象)。

另一个是只有在request.user 不是匿名的情况下才走这条路,即:

if not isinstance(request.user, AnonymousUser):
    data = get_messages(request.user.userprofile)

【讨论】:

以上是关于使用 ghost.py 抓取页面时出错的主要内容,如果未能解决你的问题,请参考以下文章

Ghost.py 的 Python 导入错误

索引 nutch 抓取的 solr 数据时出错

Selenium + PhantomJS打开的网页与常规浏览器不同

如何用python的selenium提取页面所有资源加载的链接

Python网页抓取 - 当页面通过JS加载内容时如何获取资源?

G-Ghost-RegNet实战:使用G-Ghost-RegNet实现图像分类任务(一)