代码之家 › 专栏 › 技术社区 › Madhav Shenoy

将数据从LiteDb查询复制到列表会导致System.InvalidOperationException

litedb .net-core

0

Madhav Shenoy · 技术社区 · 7 年前

我一直在玩LiteDb来提高我的数据源的性能。feed在一个Azure数据库上运行,为了减少我们使用的DTU和连接的数量(我正在并行forach循环中执行一些逻辑),我从数据库中提取数据,将其放入LiteDb索引中,然后围绕它运行我的逻辑。我的大部分场景都很好用。我首先执行insert批量添加索引,然后以只读模式重新打开litdb并运行查询

using (var db = new LiteDatabase("SiloLite.db"))
                {
                    db.DropCollection("SiloProduct");
                    var products = db.GetCollection<SiloProduct>("SiloProduct");
                    products.InsertBulk(_siloproductRepository.Table.ToList());
                    products.EnsureIndex(x => x.Sku);
                    db.Engine.EnsureIndex("SiloProduct", "UniqueProdId", "$.Sku+';'+$.ParentProductId", true);
                }
using (var db = new LiteDatabase($"Filename={AppDomain.CurrentDomain.BaseDirectory}\\SiloLite.db;mode=ReadOnly"))
                {
                    var products = db.GetCollection<SiloProduct>("SiloProduct");
                    var manufacturers = db.GetCollection<SiloManufacturer>("SiloManufacturer");
                    var productspecificationmap =
                        db.GetCollection<SiloProductSpecificationAttributeMapping>(
                            "SiloProductSpecificationAttributeMapping");
                    var specificationAttributes =
                        db.GetCollection<SiloSpecificationAttribute>("SiloSpecificationAttribute");
                    var specificationAttributeOptions =
                        db.GetCollection<SiloSpecificationAttributeOption>("SiloSpecificationAttributeOption");

                    Parallel.ForEach(katartlist, KatartItem =>
                    {
                        SaveData(KatartItem, specificationAttributes, specificationAttributeOptions, productspecificationmap,
                            manufacturers, products);
                    });
                    _logger.Information("Completed updating Product Specifications");
                }

它在除一种情况外的所有情况下都非常有效。如果我尝试运行一个查询,然后尝试将该数据保存到另一个集合LiteDb抱怨。这就是我想做的

var currentspecOptCollection = new List<SiloProductSpecificationAttributeMapping>(liteProdSpecCollection.Find(x => x.ProductId.Equals(productId)).ToList());
                    var specstoDelete = currentspecOptCollection
                                                          .Where(x => !specificationattributeoptionlist.Contains(x.SpecificationAttributeOptionId))
                                                          .ToList();
                    if (specstoDelete.Any())
                    {
                        foreach (var specattr in specstoDelete)
                        {
                            _specattrDeleteList.Add(specattr);
                        }

                    }

这个逻辑在Parallel中执行。Foreach和LiteDb抛出这个错误

System.InvalidOperationException: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.
   at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   at System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value)
   at LiteDB.CacheService.GetPage(UInt32 pageID)
   at LiteDB.PageService.GetPage[T](UInt32 pageID)
   at LiteDB.QueryCursor.Fetch(TransactionService trans, DataService data, BsonReader bsonReader)
   at LiteDB.LiteEngine.Find(String collection, Query query, Int32 skip, Int32 limit)+MoveNext()
   at LiteDB.LiteEngine.Find(String collection, Query query, String[] includes, Int32 skip, Int32 limit)+MoveNext()
   at LiteDB.LiteCollection`1.Find(Query query, Int32 skip, Int32 limit)+MoveNext()
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
   at CreateOrUpdateProductSpecifications(KatartItem katartItem, VarData vd, SiloManufacturer manufacturer, SiloProduct product, LiteCollection`1 liteSpecCollection, LiteCollection`1 liteSpecOptCollection, LiteCollection`1 liteProdSpecCollection) in *** 1399

0 回复 | 直到 7 年前

1

Muhammad Ali Nazar 6 年前

public static async Task LoadDataFromDB()
{
    var tasks = new List<Task>();

    using (var db = new LiteDatabase(fullDbPath))
    {
        var bookChapterCollection = db.GetCollection<Chapter>("bookData");
        foreach (var volume in Volume.ListOfVolumes)
        {
            tasks.Add(Task.Factory.StartNew(() =>
            {
                var volumeChapterFromDB = bookChapterCollection.Find(x => x.chapterVolume == volume.volume).toList();
                return volumeChapterFromDB;
            }
            ,TaskCreationOptions.LongRunning)
           .ContinueWith(task => {
                BookChaptersBag.Add(task.Result);
                return task.Result;
            })
            );
        }
        await Task.WhenAll(tasks).ContinueWith(task =>
        {
            BookChaptersBag.CompleteAdding();
        });
    }
}

这里的书章袋是 BlockingCollection<List<Chapter>> 它是System.Collections.Concurrent命名空间中的线程安全集合。在像上面那样编写了并行执行之后,我不再得到异常。

希望这有帮助!