mongoDB：从 pymongo 查询时失败分组
Posted
技术标签:
【中文标题】mongoDB：从 pymongo 查询时失败分组【英文标题】：mongoDB: group by failing while querying from pymongo 【发布时间】：2011-11-17 00:13:41 【问题描述】：这是我正在做的事情：
>>> import pymongo
>>> con = pymongo.Connection('localhost',12345)
>>> db = con['staging']
>>> coll = db['contract']
>>> result = coll.group(['asset_id'], None, 'list': [], 'function(obj, prev) prev.list.push(obj)')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.3-fat/egg/pymongo/collection.py", line 908, in group
File "build/bdist.macosx-10.3-fat/egg/pymongo/database.py", line 340, in command
File "build/bdist.macosx-10.3-fat/egg/pymongo/helpers.py", line 126, in _check_command_response
pymongo.errors.OperationFailure: command SON([('group', '$reduce': Code('function(obj, prev) prev.list.push(obj)', ), 'ns': u'contract', 'cond': None, 'key': 'asset_id': 1, 'initial': 'list': [])]) failed: exception: BufBuilder grow() > 64MB
我在 mongod 日志上看到的内容如下
Wed Nov 16 16:05:55 [conn209] Assertion: 13548:BufBuilder grow() > 64MB 0x10008de9b 0x100008d89 0x100151e72 0x100152712 0x100151954 0x100152712 0x100151954 0x100152712 0x100152e7b 0x100152f0c 0x10013b1d9 0x1003706bf 0x10037204c 0x10034c4d6 0x10034d877 0x100180cc4 0x100184649 0x1002b9e89 0x1002c3f18 0x100433888 0 mongod 0x000000010008de9b _ZN5mongo11msgassertedEiPKc + 315 1 mongod 0x0000000100008d89 _ZN5mongo10BufBuilder15grow_reallocateEv + 73
2 mongod 0x0000000100151e72 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 2962
3 mongod 0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
4 mongod 0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
5 mongod 0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
6 mongod 0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
7 mongod 0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
8 mongod 0x0000000100152e7b _ZN5mongo9Convertor8toObjectEl + 139
9 mongod 0x0000000100152f0c _ZN5mongo7SMScope9getObjectEPKc + 92
10 mongod 0x000000010013b1d9 _ZN5mongo11PooledScope9getObjectEPKc + 25
11 mongod 0x00000001003706bf _ZN5mongo12GroupCommand5groupESsRKSsRKNS_7BSONObjES3_SsSsPKcS3_SsRSsRNS_14BSONObjBuilderE + 3551
12 mongod 0x000000010037204c _ZN5mongo12GroupCommand3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb + 3676
13 mongod 0x000000010034c4d6 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 1350
14 mongod 0x000000010034d877 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 2151
15 mongod 0x0000000100180cc4 _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 52
16 mongod 0x0000000100184649 _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 10585
17 mongod 0x00000001002b9e89 _ZN5mongo13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 569
18 mongod 0x00000001002c3f18 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 1528
19 mongod 0x0000000100433888 _ZN5mongo10connThreadEPNS_13MessagingPortE + 616
Wed Nov 16 16:05:55 [conn209] query staging.$cmd ntoreturn:1 command: group: $reduce: CodeWScope( function(obj, prev) prev.list.push(obj), ), ns: "contract", cond: null, key: asset_id: 1 , initial: list: reslen:111 1006ms
我对pymongo和Mongodb都很陌生，不知道如何解决这个问题，请帮忙
谢谢
【问题讨论】：
【参考方案1】：堆栈跟踪的相关部分是：
exception: BufBuilder grow() > 64MB
基本上，Mongo 不允许您拥有任何大于 64MB 的文档。请参阅this question 了解更多详情（从那时起，大小限制已提高到 64MB。）
我不确定您要对该查询做什么。看起来您想要获取每个asset_id 的对象列表。但是，您的集合将超过容量，因为您永远不会区分组中的对象。尝试将您的initial 设置为'asset_id': '', 'objects': [] 和reduce 设置为function(obj, prev) prev.asset_id = obj.asset_id; prev.objects.push(obj)，尽管有更有效的方法来执行此查询。
或者，如果您要获取与 ID 匹配的所有文档，请尝试：
coll.find('asset_id': whatevs)
如果您尝试获取对象的数量，请尝试以下操作：
coll.group(
['asset_id'], None, 'asset_id': '', 'count': 0,
'function(obj, prev) prev.asset_id = obj.asset_id; prev.count += obj.count'
)
【讨论】：
以上是关于mongoDB：从 pymongo 查询时失败分组的主要内容，如果未能解决你的问题，请参考以下文章
使用 Pymongo 从 Mongodb 中的 Sql 转换“distinct on”查询
将日期对象放入 MongoDB，使用 pymongo 查询时返回浮点数