代码之家 › 专栏 › 技术社区 › maslick

将多个时间序列数据集合并为一个

time-series kotlin

maslick · 技术社区 · 8 年前

给定几组时间戳数据,如何将它们合并为一组?

假设,我有一个由以下数据结构(Kotlin)表示的数据集:

data class Data(
  val ax: Double?, 
  val ay: Double?, 
  val az: Double?, 
  val timestamp: Long
)

ax,ay,az-各轴上的加速度

时间戳-unix时间戳

Ax:
+-----+------+------+-----------+
| ax  |  ay  |  az  | timestamp |
+-----+------+------+-----------+
| 0.0 | null | null |         0 |
| 0.1 | null | null |        50 |
| 0.2 | null | null |       100 |
+-----+------+------+-----------+

Ay:
+------+-----+------+-----------+
|  ax  | ay  |  az  | timestamp |
+------+-----+------+-----------+
| null | 1.0 | null |        10 |
| null | 1.1 | null |        20 |
| null | 1.2 | null |        30 |
+------+-----+------+-----------+

Az:
+------+------+-----+-----------+
|  ax  |  ay  | az  | timestamp |
+------+------+-----+-----------+
| null | null | 2.0 |        20 |
| null | null | 2.1 |        40 |
| null | null | 2.2 |        60 |
+------+------+-----+-----------+

算法将产生:

+------+------+------+-----------+
| ax   |  ay  |  az  | timestamp |
+------+------+------+-----------+    
| 0.0  | null | null |         0 |
| 0.0  | 1.0  | null |        10 |
| 0.0  | 1.1  | 2.0  |        20 |
| 0.0  | 1.2  | 2.0  |        30 |
| 0.0  | 1.2  | 2.1  |        40 |
| 0.1  | 1.2  | 2.1  |        50 |
| 0.1  | 1.2  | 2.2  |        60 |
| 0.2  | 1.2  | 2.2  |       100 |
+------+------+------+-----------+

因此,为了将三个数据集合并为一个,我:

将Ax、Ay和Az放在一个列表中:

val united: List<Data> = arrayListOf<Data>()
united.addAll(Ax)
united.addAll(Ay)
united.addAll(Az)

按时间戳对结果列表排序:
```
united.sortBy { it.timestamp }
```

var tempAx: Double? = null
var tempAy: Double? = null
var tempAz: Double? = null

for (i in 1 until united.size) {
    val curr = united[i]
    val prev = united[i-1]

    if (curr.ax == null) {
        if (prev.ax != null) {
            curr.ax = prev.ax
            tempAx = prev.ax
        }
        else curr.ax = tempAx
    }

    if (curr.ay == null) {
        if (prev.ay != null) {
            curr.ay = prev.ay
            tempAy = prev.ay
        }
        else curr.ay = tempAy
    }

    if (curr.az == null) {
        if (prev.az != null) {
            curr.az = prev.az
            tempAz = prev.az
        }
        else curr.az = tempAz
    }
}

删除重复的行(具有相同的时间戳):
```
return united.distinctBy { it.timestamp }
```

上面的方法可以通过一次合并两个列表来改进,我也许可以为此创建一个函数。有没有更好的办法来解决这个问题?有什么想法吗?谢谢。

2 回复 | 直到 8 年前

Roland 8 年前

我想你的 Data var 而不是 val

// your tempdata containing the default (starting) values:
val tempData = Data(0.0, 0.0, 0.0, 0L)

fun extract(dataList: List<Data>, prop: KMutableProperty1<Data, Double?>) = 
   // find the first non null value for the given property
   dataList.firstOrNull { prop(it) != null }
           // extract that property
           ?.let(prop)
           // set the extracted value in our tempData so that it can reused if a null value is retrieved in future
           ?.also { prop.set(tempData, it) }
           // if the above didn't return a value, use the last one set into tempData
           ?: prop(tempData)

val mergedData = /* your united.addAll */ (Ax + Ay + Az)
             .groupBy { it.timestamp }
             // your sort by timestamp
             .toSortedMap()
             .map {(timestamp, dataList) ->
                 Data(extract(dataList, Data::ax),
                         extract(dataList, Data::ay),
                         extract(dataList, Data::az),
                         timestamp
                 )

很难想出更好的方法,因为您的主要条件(默认为最后解析的值)实际上会迫使您对数据集进行排序并保存一个(或多个)临时变量。

不用为指数操心
不需要从返回的列表中删除任何重复项(不需要 distinctBy
而 extract

也许通过重构整体也变得更具可读性。

正如您所说,您希望它能够轻松地移植到Java,下面是一个可能的Java重写:

Map<Long, List<Data>> unitedList = Stream.concat(Stream.concat(Ax.stream(), Ay.stream()), Az.stream())
            .collect(Collectors.groupingBy(Data::getTimestamp));

List<Data> mergedData = unitedList.keySet().stream().sorted()
            .map(key -> {
                List<Data> dataList = unitedList.get(key);
                return new Data(extract(dataList, Data::getAx, Data::setAx),
                        extract(dataList, Data::getAy, Data::setAy),
                        extract(dataList, Data::getAz, Data::setAz),
                        key);
            }).collect(Collectors.toList());

以及可能看起来像:

Double extract(List<Data> dataList, Function<Data, Double> getter, BiConsumer<Data, Double> setter) {
    Optional<Double> relevantProperty = dataList.stream()
            .map(getter)
            .filter(Objects::nonNull)
            .findFirst();
    if (relevantProperty.isPresent()) {
        setter.accept(tempData, relevantProperty.get());
        return relevantProperty.get();
    } else {
        return getter.apply(tempData);
    }
}

基本上是相同的机制。

maslick 8 年前

所以现在我用这个方法:

data class Data(
        var ax: Double?,
        var ay: Double?,
        var az: Double?,
        val timestamp: Long
)

fun mergeDatasets(Ax: List<Data>, Ay: List<Data>, Az: List<Data>): List<Data> {
    val united = mutableListOf<Data>()
    united.addAll(Ax)
    united.addAll(Ay)
    united.addAll(Az)

    united.sortBy { it.timestamp }

    var tempAx: Double? = null
    var tempAy: Double? = null
    var tempAz: Double? = null

    for (i in 1 until united.size) {
        val curr = united[i]
        val prev = united[i-1]

        if (curr.ax == null) {
            if (prev.ax != null) {
                curr.ax = prev.ax
                tempAx = prev.ax
            }
            else curr.ax = tempAx
        }

        if (curr.ay == null) {
            if (prev.ay != null) {
                curr.ay = prev.ay
                tempAy = prev.ay
            }
            else curr.ay = tempAy
        }

        if (curr.az == null) {
            if (prev.az != null) {
                curr.az = prev.az
                tempAz = prev.az
            }
            else curr.az = tempAz
        }

        if (curr.timestamp == prev.timestamp) {
            prev.ax = curr.ax
            prev.ay = curr.ay
            prev.az = curr.az
        }
    }

    return united.distinctBy { it.timestamp }
}