• 关于mongo的$push和$addToSet的内存占用报错


    在使用mongo的聚合管道的时候会发现如果涉及的文档数过多,出现如下报错

    com.mongodb.MongoCommandException: Command failed with error 146 (ExceededMemoryLimit): '$push used too much memory and cannot spill to disk. Memory limit: 104857600 bytes' on server 192.168.2.234:27017. The full response is {"ok": 0.0, "errmsg": "$push used too much memory and cannot spill to disk. Memory limit: 104857600 bytes", "code": 146, "codeName": "ExceededMemoryLimit"}
      at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:175)
      at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:302)
      at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:258)
      at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
      at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:500)
      at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:71)
      at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:224)
      at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:202)
      at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:118)
      at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:110)
      at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343)
      at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334)
      at com.mongodb.internal.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:220)
      at com.mongodb.internal.operation.CommandOperationHelper$5.call(CommandOperationHelper.java:206)
      at com.mongodb.internal.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:462)
      at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:203)
      at com.mongodb.internal.operation.AggregateOperationImpl.execute(AggregateOperationImpl.java:189)
      at com.mongodb.internal.operation.AggregateOperation.execute(AggregateOperation.java:296)
      at com.mongodb.internal.operation.AggregateOperation.execute(AggregateOperation.java:41)
      at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:190)
      at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:135)
      at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:92)
      at com.mongodb.client.internal.MongoIterableImpl.forEach(MongoIterableImpl.java:121)
      at com.mongodb.client.internal.MongoIterableImpl.into(MongoIterableImpl.java:130)

    经过网上查询,源赖式,佐田,mongo把这两个聚合操作里面的内存限制在了100M,具体不太清楚为啥限制,默认是100M吧,但是在之后mongodb版本里面可以进行设置,通过mongo shell如下操作即可,命令行的话,通过setParameter

    > newLimit = 100 * 1024 * 1024;
    > db.adminCommand({setParameter: 1, internalQueryMaxAddToSetBytes: newLimit})
     
    不过,不太建议改这个东西,没有意义,因为mongo的文档大小有限制,为16M,就算你在聚合管道的某个阶段给$push操作数组的内存
    大小放开,但是,最后如果输出的文档包含此过大数组,还是会报错
    com.mongodb.MongoCommandException: Command failed with error 10334 (BSONObjectTooLarge): 'BSONObj size: 23537822 (0x167289E) is invalid. Size must be between 0 and 16793600(16MB) First element:
    后面太长了,就不贴了,所以一般改了也没啥用吧,除非确实是中间结果需要处理。
    参考链接:
    https://stackoverflow.com/questions/63498668/how-to-set-internalquerymaxaddtosetbytes-in-mongo-db-version-3-6-9
     
     
  • 相关阅读:
    【leetcode】对称二叉树
    【leetcode】判断回文数
    053686
    053685
    053684
    053683
    053682
    053681
    053680
    053477
  • 原文地址:https://www.cnblogs.com/marshwinter/p/14304335.html
Copyright © 2020-2023  润新知