代码之家  ›  专栏  ›  技术社区  ›  Eric Green

在时间序列中分解

  •  2
  • Eric Green  · 技术社区  · 7 年前

    我有一个数据集,我想形象化的整体和几个不同的变量分类。我创建了一个flexdashboard,其中包含一个toy-shinny应用程序来选择分解类型,并使用工作代码来绘制正确的子集。

    我的方法是重复的,这对我来说是一个暗示,我错过了一个更好的方法来做到这一点。让我感到困惑的是需要按日期计数并扩展矩阵。我不知道如何得到一个管道每周计数组。我把它分为几个步骤并结合起来。

    思想?

    (附:我是在星期五问这个问题的 RStudio Community 但我觉得这更像是 SO question ". 我没有权限从RSC中删除它,所以很抱歉交叉发布。)

    ---
    title: "test"
    output: 
      flexdashboard::flex_dashboard:
        theme: bootstrap
    runtime: shiny
    ---
    
    ```{r setup, include=FALSE}
      library(flexdashboard)
      library(tidyverse)
      library(tibbletime)
      library(dygraphs)
      library(magrittr)
      library(xts)
    ```
    
    ```{r global, include=FALSE}
      set.seed(1)
      dat <- data.frame(date = seq(as.Date("2018-01-01"), 
                                   as.Date("2018-06-30"), 
                                   "days"),
                        sex = sample(c("male", "female"), 181, replace=TRUE),
                        lang = sample(c("english", "spanish"), 181, replace=TRUE),
                        age = sample(20:35, 181, replace=TRUE))
      dat <- sample_n(dat, 80)
    ```
    
    Sidebar {.sidebar}
    =====================================
    
    ```{r}
      radioButtons("diss", label = "Disaggregation",
        choices = list("All" = 1, "By Sex" = 2, "By Language" = 3), 
        selected = 1)
    ```
    
    Page 1
    =====================================
    
    ```{r}
    # all
      all <- reactive(
      dat %>%  
        mutate(new = 1) %>%
        arrange(date) %>%
      # time series analysis
        as_tbl_time(index = date) %>% # convert to tibble time object
        select(date, new) %>%
        collapse_by('1 week', side="start", clean=TRUE) %>%
        group_by(date) %>%
        mutate(total = sum(new, na.rm=TRUE)) %>% 
        distinct(date, .keep_all = TRUE) %>% 
        ungroup() %>%
      # expand matrix to include weeks without data
        complete(date = seq(date[1],
                            date[length(date)],
                            by = "1 week"),
                 fill = list(total = 0)) 
      )
    
    # males only
      males <- reactive(
      dat %>%  
        filter(sex=="male") %>%
        mutate(new = 1) %>%
        arrange(date) %>%
      # time series analysis
        as_tbl_time(index = date) %>%
        select(date, new) %>%
        collapse_by('1 week', side="start", clean=TRUE) %>%
        group_by(date) %>%
        mutate(total_m = sum(new, na.rm=TRUE)) %>% 
        distinct(date, .keep_all = TRUE) %>% 
        ungroup() %>%
      # expand matrix to include weeks without data
        complete(date = seq(date[1],
                            date[length(date)],
                            by = "1 week"),
                 fill = list(total_m = 0)) 
      )
    
    # females only
      females <- reactive(
      dat %>%  
        filter(sex=="female") %>%
        mutate(new = 1) %>%
        arrange(date) %>%
      # time series analysis
        as_tbl_time(index = date) %>%
        select(date, new) %>%
        collapse_by('1 week', side="start", clean=TRUE) %>%
        group_by(date) %>%
        mutate(total_f = sum(new, na.rm=TRUE)) %>% 
        distinct(date, .keep_all = TRUE) %>% 
        ungroup() %>%
      # expand matrix to include weeks without data
        complete(date = seq(date[1],
                            date[length(date)],
                            by = "1 week"),
                 fill = list(total_f = 0)) 
      )
    
    # english only
      english <- reactive(
      dat %>%  
        filter(lang=="english") %>%
        mutate(new = 1) %>%
        arrange(date) %>%
      # time series analysis
        as_tbl_time(index = date) %>%
        select(date, new) %>%
        collapse_by('1 week', side="start", clean=TRUE) %>%
        group_by(date) %>%
        mutate(total_e = sum(new, na.rm=TRUE)) %>% 
        distinct(date, .keep_all = TRUE) %>% 
        ungroup() %>%
      # expand matrix to include weeks without data
        complete(date = seq(date[1],
                            date[length(date)],
                            by = "1 week"),
                 fill = list(total_e = 0)) 
      )
    
    # spanish only
      spanish <- reactive(
      dat %>%  
        filter(lang=="spanish") %>%
        mutate(new = 1) %>%
        arrange(date) %>%
      # time series analysis
        as_tbl_time(index = date) %>%
        select(date, new) %>%
        collapse_by('1 week', side="start", clean=TRUE) %>%
        group_by(date) %>%
        mutate(total_s = sum(new, na.rm=TRUE)) %>% 
        distinct(date, .keep_all = TRUE) %>% 
        ungroup() %>%
      # expand matrix to include weeks without data
        complete(date = seq(date[1],
                            date[length(date)],
                            by = "1 week"),
                 fill = list(total_s = 0)) 
      )
    
    # combine
    
      totals <- reactive({
    
      all <- all()
      females <- females()
      males <- males()
      english <- english()
      spanish <- spanish()
    
      all %>%
        select(date, total) %>%
        full_join(select(females, date, total_f), by = "date") %>%
        full_join(select(males, date, total_m), by = "date") %>%
        full_join(select(english, date, total_e), by = "date") %>%
        full_join(select(spanish, date, total_s), by = "date") 
      })
    
    # convert to xts
      totals_ <- reactive({
        totals <- totals()
        xts(totals, order.by = totals$date)
      })
    
    # plot
      renderDygraph({
    
      totals_ <- totals_()
    
      if (input$diss == 1) {
      dygraph(totals_[, "total"],
              main= "All") %>%
        dySeries("total", label = "All") %>%
        dyRangeSelector() %>%
        dyOptions(useDataTimezone = FALSE,
                  stepPlot = TRUE,
                  drawGrid = FALSE,
                  fillGraph = TRUE) 
      } else if (input$diss == 2) {
        dygraph(totals_[, c("total_f", "total_m")],
                main = "By sex") %>%
        dyRangeSelector() %>%
        dySeries("total_f", label = "Female") %>%
        dySeries("total_m", label = "Male") %>%
        dyOptions(useDataTimezone = FALSE,
                  stepPlot = TRUE,
                  drawGrid = FALSE,
                  fillGraph = TRUE) 
      } else {
        dygraph(totals_[, c("total_e", "total_s")],
                main = "By language") %>%
        dyRangeSelector() %>%
        dySeries("total_e", label = "English") %>%
        dySeries("total_s", label = "Spanish") %>%
        dyOptions(useDataTimezone = FALSE,
                  stepPlot = TRUE,
                  drawGrid = FALSE,
                  fillGraph = TRUE)
      }
      })
    ```
    

    更新:

    @jonspring建议编写一个函数来减少一些重复(应用于下面),这是一个很好的改进。不过,基本方法是相同的。分段、计算、合并、绘图。有没有办法做到这一点而不拆散和重新组装?

    ---
    title: "test"
    output: 
      flexdashboard::flex_dashboard:
        theme: bootstrap
    runtime: shiny
    ---
    
    ```{r setup, include=FALSE}
      library(flexdashboard)
      library(tidyverse)
      library(tibbletime)
      library(dygraphs)
      library(magrittr)
      library(xts)
    ```
    
    ```{r global, include=FALSE}
    # generate data
      set.seed(1)
      dat <- data.frame(date = seq(as.Date("2018-01-01"), 
                                   as.Date("2018-06-30"), 
                                   "days"),
                        sex = sample(c("male", "female"), 181, replace=TRUE),
                        lang = sample(c("english", "spanish"), 181, replace=TRUE),
                        age = sample(20:35, 181, replace=TRUE))
      dat <- sample_n(dat, 80)
    
    # Jon Spring's function
      prep_dat <- function(filtered_dat, col_name = "total") {
      filtered_dat %>%
        mutate(new = 1) %>%
        arrange(date) %>%
      # time series analysis
        tibbletime::as_tbl_time(index = date) %>% # convert to tibble time object
        select(date, new) %>%
        tibbletime::collapse_by("1 week", side = "start", clean = TRUE) %>%
        group_by(date) %>%
        mutate(total = sum(new, na.rm = TRUE)) %>%
        distinct(date, .keep_all = TRUE) %>%
        ungroup() %>%
        # expand matrix to include weeks without data
        complete(
          date = seq(date[1], date[length(date)], by = "1 week"),
          fill = list(total = 0)
        )
      }
    ```
    
    Sidebar {.sidebar}
    =====================================
    
    ```{r}
      radioButtons("diss", label = "Disaggregation",
        choices = list("All" = 1, "By Sex" = 2, "By Language" = 3), 
        selected = 1)
    ```
    
    Page 1
    =====================================
    
    ```{r}
    # all
      all <- reactive(
      prep_dat(dat) 
      )
    
    # males only
      males <- reactive(
      prep_dat(
        dat %>% 
        filter(sex == "male")
      ) %>% 
        rename("total_m" = "total")
      )
    
    # females only
      females <- reactive(
      prep_dat(
        dat %>% 
        filter(sex == "female")
      ) %>% 
        rename("total_f" = "total")
      )
    
    # english only
      english <- reactive(
      prep_dat(
        dat %>% 
        filter(lang == "english")
      ) %>% 
        rename("total_e" = "total")
      )
    
    # spanish only
      spanish <- reactive(
      prep_dat(
        dat %>% 
        filter(lang == "spanish")
      ) %>% 
        rename("total_s" = "total")
      )
    
    # combine
    
      totals <- reactive({
    
      all <- all()
      females <- females()
      males <- males()
      english <- english()
      spanish <- spanish()
    
      all %>%
        select(date, total) %>%
        full_join(select(females, date, total_f), by = "date") %>%
        full_join(select(males, date, total_m), by = "date") %>%
        full_join(select(english, date, total_e), by = "date") %>%
        full_join(select(spanish, date, total_s), by = "date") 
      })
    
    # convert to xts
      totals_ <- reactive({
        totals <- totals()
        xts(totals, order.by = totals$date)
      })
    
    # plot
      renderDygraph({
    
      totals_ <- totals_()
    
      if (input$diss == 1) {
      dygraph(totals_[, "total"],
              main= "All") %>%
        dySeries("total", label = "All") %>%
        dyRangeSelector() %>%
        dyOptions(useDataTimezone = FALSE,
                  stepPlot = TRUE,
                  drawGrid = FALSE,
                  fillGraph = TRUE) 
      } else if (input$diss == 2) {
        dygraph(totals_[, c("total_f", "total_m")],
                main = "By sex") %>%
        dyRangeSelector() %>%
        dySeries("total_f", label = "Female") %>%
        dySeries("total_m", label = "Male") %>%
        dyOptions(useDataTimezone = FALSE,
                  stepPlot = TRUE,
                  drawGrid = FALSE,
                  fillGraph = TRUE) 
      } else {
        dygraph(totals_[, c("total_e", "total_s")],
                main = "By language") %>%
        dyRangeSelector() %>%
        dySeries("total_e", label = "English") %>%
        dySeries("total_s", label = "Spanish") %>%
        dyOptions(useDataTimezone = FALSE,
                  stepPlot = TRUE,
                  drawGrid = FALSE,
                  fillGraph = TRUE)
      }
      })
    ```
    
    3 回复  |  直到 7 年前
        1
  •  2
  •   Jon Spring    7 年前

    谢谢你解释更多关于你的目标。我认为@simon-s-a建议的方法会简化事情。如果我们可以动态地运行分组,并对其进行结构化,这样就不需要事先知道这些分组中可能的组件,那么维护起来就会容易得多。

    1. 然后我用 padr::pad 填充中间丢失的时间行,并用零替换所有NA。

    2. xts 对象并将其输入dygraph,dygraph似乎可以自动处理多个列。

    ---
    title: "test"
    output: 
      flexdashboard::flex_dashboard:
        theme: bootstrap
    runtime: shiny
    ---
    
    ```{r setup, include=FALSE}
    library(flexdashboard)
    library(tidyverse)
    library(tibbletime)
    library(dygraphs)
    library(magrittr)
    library(xts)
    ```
    
    ```{r global, include=FALSE}
    # generate data
    set.seed(1)
    dat <- data.frame(date = seq(as.Date("2018-01-01"), 
                                 as.Date("2018-06-30"), 
                                 "days"),
                      sex = sample(c("male", "female"), 181, replace=TRUE),
                      lang = sample(c("english", "spanish"), 181, replace=TRUE),
                      age = sample(20:35, 181, replace=TRUE))
    dat <- dplyr::sample_n(dat, 80)
    ```
    
    Sidebar {.sidebar}
    =====================================
    
    ```{r}
    
    radioButtons("diss", label = "Disaggregation",
                 choices = list("All" = "Total",
                                "By Sex" = "sex",
                                "By Language" = "lang"), 
                 selected = "Total")
    ```
    
    Page 1
    =====================================
    
    ```{r plot}
    
    renderDygraph({
      grp_col <- rlang::sym(input$diss) # This converts the input selection to a symbol
    
      dat %>%
        mutate(Total = 1) %>% # This is a hack to let us "group" by Total -- all one group
    
        # Here's where we unquote the symbol so that dplyr can use it 
        #   to refer to a column. In this case I make a dummy column 
        #   that's a copy of whatever column we want to group
        mutate(my_group = !!grp_col) %>%
    
        # Now we make a group for every existing combination of week 
        #   (using lubridate::floor_date) and level of our grouping column,
        #   count how many rows in each group, and spread that to wide format.
        group_by(date = lubridate::floor_date(date, "1 week"), my_group) %>%
        count() %>% spread(my_group, n) %>% ungroup() %>%
    
        # padr:pad() fills in any missing weeks in the sequence with new rows
        #   Then we replace all the NA's with zeroes.
        padr::pad() %>% replace(is.na(.), 0) %>%
    
        # Finally we can convert to xts and feed the wide table into digraph.
        xts::xts(order.by = .$date) %>%
        dygraph() %>%
        dyRangeSelector() %>%
        dyOptions(
          useDataTimezone = FALSE, stepPlot = TRUE,
          drawGrid = FALSE, fillGraph = TRUE
        )
    })
    ```
    
        2
  •  1
  •   Jon Spring    7 年前

    这是一个创建函数的好地方,可以缩短代码并减少出错的可能性。

    http://r4ds.had.co.nz/functions.html

    dplyr 通常需要涉入一个名为tidyeval的框架,这个框架非常强大,但可能会很吓人。 https://dplyr.tidyverse.org/articles/programming.html

    (这里有一种避开潮水的替代方法: https://cran.r-project.org/web/packages/seplyr/vignettes/using_seplyr.html )

    prep_dat <- function(filtered_dat, col_name = "total") {
      filtered_dat %>%
        mutate(new = 1) %>%
        arrange(date) %>%
      # time series analysis
      tibbletime::as_tbl_time(index = date) %>% # convert to tibble time object
        select(date, new) %>%
        tibbletime::collapse_by("1 week", side = "start", clean = TRUE) %>%
        group_by(date) %>%
        mutate(total = sum(new, na.rm = TRUE)) %>%
        distinct(date, .keep_all = TRUE) %>%
        ungroup() %>%
        # expand matrix to include weeks without data
        complete(
          date = seq(date[1], date[length(date)], by = "1 week"),
          fill = list(total = 0)
        )
    }
    

    然后您可以使用筛选数据和total列的名称来调用它。这个片段应该可以替换您当前使用的~20行:

    males <- prep_dat(dat_fake %>% 
      filter(sex == "male")) %>% 
      rename("total_m" = "total")
    

    dat_fake <- tibble(
      date = as.Date("2018-01-01") + runif(500, 0, 100),
      new  = runif(500, 0, 100),
      sex  = sample(c("male", "female"), 
                    500, replace = TRUE),
      lang = sample(c("english", "french", "spanish", "portuguese", "tagalog"), 
                    500, replace = TRUE)
    )
    
        3
  •  1
  •   Simon.S.A.    7 年前

    我想你可以通过改变准备的顺序来有所收获。现在,你的应用程序的流程大约是:

    请考虑:

    这将利用Shiny的反应性(重新)准备所需的数据,以响应用户选择的变化。

    通过代码片段的方式(对不起,我对 flexdashboard tibbletime 以确保此代码运行,但我希望它足以突出显示方法):

    控件选择要关注的列(注意 "All" = "'1'"

    radioButtons("diss", label = "Disaggregation",
                 choices = list("All" = "'1'",
                                "By Sex" = "sex",
                                "By Language" = "lang",
                                "By other" = "column_name_of_'other'"), 
                 selected = 1)
    

    然后在您的分组中使用这个,只准备当前可视化所需的数据(您需要调整@Jon\u Spring建议的函数,以响应前面的分组):

    preped_dat = reactive({
      dat %>%
        group_by_(input$diss) %>%
        # etc
    })
    

    renderDygraph({
      totals = preped_data()
      dygraph(totals) %>%
          dySeries("total", label = ) %>%
          dyRangeSelector()
    })
    

    group_by 你可以用 group_by_ 如果所有参数都是文本字符串,或者 group_by(!! sym(input$diss), other_column_name) 如果要将控件输入的文本字符串与其他列名混合使用。

    这种方法改变的一个可能的缺点是,如果数据集很大,则在交互过程中会降低响应速度。目前的方法先进行所有的计算,然后对每个选择进行最小的计算——如果您有大量的处理,这可能是更好的方法。我建议的方法将有最小的前期处理和适度的计算每个选择。