它带我四处转转
6.5至7小时
在从的数据帧插入批量记录时
550万记录
对我的
mongodb数据库
通过
django模型
具有
12-GB内存
.下面是引发查询的代码。我想让它更快地四处走动
30分钟到1小时
时间,可能吗?
# Split DF
listDf = []
listDf = split_dataframe(completeDF02)
## This completeDF02 is the full dataframe having 5.5M records, So I split it with having 0.1M record in one batch in listDf list
for item in listDf:
df_records = item.to_dict('records')
model_instances = [MyModel(
Field1=record['Field1'],
Field2=record['Field2'],
Field3=record['Field3'],
Field4=record['Field4'],
Field5=record['Field5'],
Field6=record['Field6'],
Field7=record['Field7'],
Field8=record['Field8'],
Field9=record['Field9'],
Field10=record['Field10'],
Field11=record['Field11'],
Field12=record['Field12'],
Field13=record['Field13'],
Field14=record['Field14'],
Field15=record['Field15'],
Field16=record['Field16'],
Field17=record['Field17'],
Field18=record['Field18'],
Field19=record['Field19'],
Field20=record['Field20'],
Field21=record['Field21'],
Field22=record['Field22'],
Field23=record['Field23'],
Field24=record['Field24'],
Field25=record['Field25']
) for record in df_records]
MyModel.objects.bulk_create(model_instances)
有没有其他方法可以进行批量插入,以提高速度和效率?如果是,请告诉我。