在Async Promise内批量加载时Grails DuplicateKeyException/NonUniqueObjectException

我有大量数据,希望使用GORM加载到数据库中。

class DbLoadingService {

    static transactional = false    
    // these are used to expedite the batch loading process
    def sessionFactory
    def propertyInstanceMap = org.codehaus.groovy.grails.plugins.DomainClassGrailsPlugin.PROPERTY_INSTANCE_MAP

    // these are example services that will assist in the parsing of the input data    
    def auxLoadingServiceA
    def auxLoadingServiceB

    def handleInputFile(String filename) {
        def inputFile = new File(filename)
        // parse each line and process according to record type
        inputFile.eachLine { line, lineNumber ->
            this.handleLine(line, lineNumber)
        }
    }


    @Transactional
    def handleLine(String line, int lineNumber) {
        // do some further parsing of the line, based on its content
        // example here is based on 1st 2 chars of line
        switch (line[0..1]) {
            case 'AA':
                auxLoadingServiceA.doSomethingWithLine(line)
                break;

            case 'BB':
                auxLoadingServiceB.doSomethingElseWithLine(line)
                break;

            default:
                break;

        }
        if (lineNumber % 100 == 0) cleanUpGorm()
    }

    def cleanUpGorm() {
        def session = sessionFactory.getCurrentSession()
        session.flush()
        session.clear()
        propertyInstanceMap.get().clear()
    }

}

class AuxLoadingServiceA {
    static transactional = false

    doSomethingWithLine(String line) {
        // do something here
    }
}

class AuxLoadingServiceB {
    static transactional = false

    doSomethingElseWithLine(String line) {
        // do something else here
    }
}

我特意使顶级服务仅针对每行的负载进行事务处理。实际上,顶层下有相当多的服务级别,而不仅仅是单个AuxA&B服务层。因此,我不希望有多层事务的开销:我认为我只需要1。

加载到DB中的数据模型包括两个具有Many/belongsTo关系的域对象。与域对象的这种交互是在子层中完成的,为了保持示例的大小可管理,我的代码中没有显示。

似乎导致问题的域对象与以下内容类似:

class Parent {
    static hasMany = [children: Child]
    static mapping = {
        children lazy: false
        cache true
    }
}

class Child {
    String someValue
    // also contains some other sub-objects

    static belongsTo = [parent : Parent]

    static mapping = {
        parent index: 'parent_idx'
        cache true
    }
}

所示的cleanupGorm()方法是必需的,否则服务会在大量的行之后完全停止。

当我启动数据库加载时,所有工作都完全按照预期进行:

// Called from with a service / controller
dbLoadingService.handleInputFile("someFile.txt")

但是,一旦我将负载移动到异步进程中,如下所示:

def promise = task {
    dbLoadingService.handleInputFile("someFile.txt")
}

我得到了DuplicateKeyException/NonUniqueObjectException:

error details: org.springframework.dao.DuplicateKeyException: A different object with the same identifier value was already associated with the session : [com.example.SampleDomainObject#1]; nested exception is org.hibernate.NonUniqueObjectException: A different object with the same identifier value was already associated with the session : [com.example.SampleDomainObject#1]

所以,我的问题是,在将大量数据异步加载到Grails DB中方面,有哪些最佳实践?为了确保内存中的对象在会话中保持一致,是否需要对刷新/清除会话进行一些操作?缓存对象时需要做什么吗?