代码之家 › 专栏 › 技术社区 › Anthony Amico

在r的循环中绑定表

Anthony Amico · 技术社区 · 6 年前

我正试图从体育参考中获取大量数据。我的编码背景非常薄弱,因为我只在几个过程中自学。我已经了解了如何使用htmltab()函数从sr中提取数据,并且可以从网站上的每个页面创建一个表。

我的问题是把最后的表格结合起来。我知道下面的代码只使用5页,使用rbind()很容易组合,但它只是一个小的测试样本。

我最终将有数千个表要合并,所以在最后手动对它们进行循环是不实际的。在循环的每一步中,是否有一种方法可以将每个新表附加到某个复合表上(或者在末尾轻松地绑定它们,而无需键入数千个表)?

或者,如果我可以将所有数据合并到一个表中,而不必首先生成1000个数据,那么看起来效率更高,但我不知道如何做到这一点(显然)。

感谢您的帮助!

(对于不熟悉sr的人,站点将其表按100个元素分组,因此i*100和粘贴到URL的第一部分)

for (i in 1:5) {
     a <- i*100
     url <- paste("https://www.sports-reference.com/cfb/play-index/pgl_finder.cgi?request=1&match=game&year_min=&year_max=&conf_id=&school_id=&opp_id=&game_type=&game_num_min=&game_num_max=&game_location=&game_result=&class=&c1stat=rush_att&c1comp=gt&c1val=0&c2stat=rec&c2comp=gt&c2val=0&c3stat=punt_ret&c3comp=gt&c3val=0&c4stat=kick_ret&c4comp=gt&c4val=0&order_by=date_game&order_by_asc=&offset=",a,sep = "")
     nam <- paste("ploop",i,sep = "")
     assign(nam,htmltab(url))
     ??????
     }

2 回复 | 直到 6 年前

jdobres 6 年前

在这种情况下,最好将结果存储在列表中,而不是与 assign . 在这里,我们将循环每次迭代的结果存储在一个列表中,然后使用 do.call 具有 rbind 要创建单个数据帧:

rm(list = ls())
library(htmltab)

tables <- list()
for (i in 1:5) {
  a <- i*100
  url <- paste("https://www.sports-reference.com/cfb/play-index/pgl_finder.cgi?request=1&match=game&year_min=&year_max=&conf_id=&school_id=&opp_id=&game_type=&game_num_min=&game_num_max=&game_location=&game_result=&class=&c1stat=rush_att&c1comp=gt&c1val=0&c2stat=rec&c2comp=gt&c2val=0&c3stat=punt_ret&c3comp=gt&c3val=0&c4stat=kick_ret&c4comp=gt&c4val=0&order_by=date_game&order_by_asc=&offset=",a,sep = "")
  tables[[i]] <- htmltab(url)
}

table.final <- do.call(rbind, tables)

str(table.final)

'data.frame':   520 obs. of  20 variables:
 $ Rk              : chr  "101" "102" "103" "104" ...
 $ Player          : chr  "Myles Gaskin" "Willie Gay" "Jake Gervase" "Kyle Gibson" ...
 $ Date            : chr  "2019-01-01" "2019-01-01" "2019-01-01" "2019-01-01" ...
 $ G#              : chr  "14" "13" "13" "13" ...
 $ School          : chr  "Washington" "Mississippi State" "Iowa" "Central Florida" ...
 $ V2              : chr  "N" "N" "N" "N" ...
 $ Opponent        : chr  "Ohio State" "Iowa" "Mississippi State" "Louisiana State" ...
 $ V2.1            : chr  "L" "L" "W" "L" ...
 $ Rushing >> Att  : chr  "24" "0" "0" "0" ...
 $ Rushing >> Yds  : chr  "121" "0" "0" "0" ...
 $ Rushing >> TD   : chr  "2" "0" "0" "0" ...
 $ Receiving >> Rec: chr  "3" "0" "0" "0" ...
 $ Receiving >> Yds: chr  "-1" "0" "0" "0" ...
 $ Receiving >> TD : chr  "0" "0" "0" "0" ...
 $ Kick Ret >> Ret : chr  "0" "0" "0" "0" ...
 $ Kick Ret >> Yds : chr  "0" "0" "0" "0" ...
 $ Kick Ret >> TD  : chr  "0" "0" "0" "0" ...
 $ Punt Ret >> Ret : chr  "0" "0" "0" "0" ...
 $ Punt Ret >> Yds : chr  "0" "0" "0" "0" ...
 $ Punt Ret >> TD  : chr  "0" "0" "0" "0" ...

José 6 年前

您还可以尝试tidyverse方式:

url <- "https://www.sports-reference.com/cfb/play-index/pgl_finder.cgi?request=1&match=game&year_min=&year_max=&conf_id=&school_id=&opp_id=&game_type=&game_num_min=&game_num_max=&game_location=&game_result=&class=&c1stat=rush_att&c1comp=gt&c1val=0&c2stat=rec&c2comp=gt&c2val=0&c3stat=punt_ret&c3comp=gt&c3val=0&c4stat=kick_ret&c4comp=gt&c4val=0&order_by=date_game&order_by_asc=&offset="

df <- purrr::map_dfr(1:5,~htmltab::htmltab(paste0(url,.x*100)))