代码之家 › 专栏 › 技术社区 › Tdebeus

从下载的文件中提取源元数据

metadata macos r

Tdebeus · 技术社区 · 7 年前

我下载了一堆pdf文件。现在我想从文件的元数据中提取下载url。如何以编程方式执行此操作?我更喜欢R的解决方案,我正在研究MacOS Mojave。

如果你想繁殖,你可以 [use this file] .

2 回复 | 直到 7 年前

hrbrmstr 7 年前

xattrs 获取所需数据的程序包:

library(xattrs) # https://gitlab.com/hrbrmstr/xattrs (not on CRAN)

xattrs::list_xattrs("~/Downloads/0.-miljoenennota.pdf")
## [1] "com.apple.metadata:kMDItemWhereFroms"
## [2] "com.apple.quarantine"

com.apple.metadata:kMDItemWhereFroms

xattrs::get_xattr(
  path = "~/Downloads/forso/0.-miljoenennota.pdf",
  name = "com.apple.metadata:kMDItemWhereFroms"
) -> from_where

from_where
## [1] "bplist00\xa2\001\002_\020}https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/begrotingen/2016/09/20/miljoenennota-2017/0.-miljoenennota.pdfP\b\v\x8b"

但是,它是二进制plist格式的。然而,既然这是“一件事”,那么 扩展属性 包有一个 read_bplist() 功能,但我们必须使用 get_xattr_raw()

xattrs::read_bplist(
  xattrs::get_xattr_raw(
    path = "~/Downloads/forso/0.-miljoenennota.pdf",
    name = "com.apple.metadata:kMDItemWhereFroms"
  )
) -> from_where

str(from_where)
## List of 1
##  $ plist:List of 1
##   ..$ array:List of 2
##   .. ..$ string:List of 1
##   .. .. ..$ : chr "https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/begrotingen/2016/09/20/miljoenennota-2017/0.-miljoenennota.pdf"
##   .. ..$ string: list()
##   ..- attr(*, "version")= chr "1.0"

丑陋的嵌套列表是真正愚蠢的二进制plist文件格式的错误,但是源URL在那里。

我们可以通过使用 lapply . 还有一个例子 in this blog post reticulate 以及一个Python包来读取二进制plist数据,而不是内置的包函数来完成这项工作(所说的内置包函数是macOS的包装器) plutil 实用程序或linux plistutil 效用;Windows用户可以切换到一个真正的操作系统,如果他们想使用该功能)。

fils <- list.files("~/Downloads/forso", pattern = "\\.pdf", full.names = TRUE)

do.call(
  rbind.data.frame,
  lapply(fils, function(.x) {

    xattrs::read_bplist(
      xattrs::get_xattr_raw(
        path = .x,
        name = "com.apple.metadata:kMDItemWhereFroms"
      )
    ) -> tmp

    from_where <- if (length(tmp$plist$array$string) > 0) {
      tmp$plist$array$string[[1]]
    } else {
      NA_character_
    }

    data.frame(
      fil = basename(.x),
      url = from_where,
      stringsAsFactors=FALSE
    )

  })
) -> files_with_meta

str(files_with_meta)
## 'data.frame': 9 obs. of  2 variables:
##  $ fil: chr  "0.-miljoenennota.pdf" "19180242-D02E-47AC-BDB3-73C22D6E1FDB.pdf" "Codebook.pdf" "Elementary-Lunch-Menu.pdf" ...
##  $ url: chr  "https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/begrotingen/2016/09/20/miljoenennota-2017/0.-miljoenennota.pdf" "http://eprint.ncl.ac.uk/file_store/production/230123/19180242-D02E-47AC-BDB3-73C22D6E1FDB.pdf" "http://apps.start.umd.edu/gtd/downloads/dataset/Codebook.pdf" "http://www.msad60.org/wp-content/uploads/2017/01/Elementary-February-Lunch-Menu.pdf" ...

IRTFM 7 年前

Ask Different 有关模拟从Terminal.app命令行选择“Get Info”的方法。

我找到了使用命令的建议 mdls 我是从一个医生那里得到的 system

system("mdls -name kMDItemWhereFroms ~/0.-miljoenennota.pdf")

#kMDItemWhereFroms = (
#   "https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/begrotingen/2016/09/20/miljoenennota-2017/0.-miljoenennota.pdf",
#    ""
#)

要将多行结果放到R中(而不是仅仅出现在控制台上),您需要添加 intern=TRUE 参数电话:

> res <- system("mdls -name kMDItemWhereFroms ~/0.-miljoenennota.pdf", intern=TRUE)
> res
[1] "kMDItemWhereFroms = ("                                                                                                                 
[2] "    \"https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/begrotingen/2016/09/20/miljoenennota-2017/0.-miljoenennota.pdf\","
[3] "    \"\""                                                                                                                              
[4] ")"                                                                                                                                     
> res[2]
[1] "    \"https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/begrotingen/2016/09/20/miljoenennota-2017/0.-miljoenennota.pdf\","

system("mdls ~/0.-miljoenennota.pdf")

#-----------
_kMDItemOwnerUserID            = 501
kMDItemAuthors                 = (
    "Tweede Kamer der Staten-Generaal"
)
kMDItemContentCreationDate     = 2018-10-08 23:45:35 +0000
kMDItemContentModificationDate = 2018-10-08 23:45:46 +0000
kMDItemContentType             = "com.adobe.pdf"
kMDItemContentTypeTree         = (
    "com.adobe.pdf",
    "public.data",
    "public.item",
    "public.composite-content",
    "public.content"
)
kMDItemCreator                 = "XPP"
kMDItemDateAdded               = 2018-10-08 23:45:46 +0000
kMDItemDisplayName             = "0.-miljoenennota.pdf"
kMDItemEncodingApplications    = (
    "Acrobat Distiller Server 8.1.0 (Pentium Linux, Built: 2007-09-07)"
)
kMDItemFSContentChangeDate     = 2018-10-08 23:45:46 +0000
kMDItemFSCreationDate          = 2018-10-08 23:45:35 +0000
kMDItemFSCreatorCode           = ""
kMDItemFSFinderFlags           = 0
kMDItemFSHasCustomIcon         = (null)
kMDItemFSInvisible             = 0
kMDItemFSIsExtensionHidden     = 0
kMDItemFSIsStationery          = (null)
kMDItemFSLabel                 = 0
kMDItemFSName                  = "0.-miljoenennota.pdf"
kMDItemFSNodeCount             = (null)
kMDItemFSOwnerGroupID          = 20
kMDItemFSOwnerUserID           = 501
kMDItemFSSize                  = 4004668
kMDItemFSTypeCode              = ""
kMDItemKind                    = "Portable Document Format (PDF)"
kMDItemLogicalSize             = 4004668
kMDItemNumberOfPages           = 196
kMDItemPageHeight              = 841.89
kMDItemPageWidth               = 595.276
kMDItemPhysicalSize            = 4005888
kMDItemSecurityMethod          = "None"
kMDItemVersion                 = "1.6"
kMDItemWhereFroms              = (
    "https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/begrotingen/2016/09/20/miljoenennota-2017/0.-miljoenennota.pdf",
    ""
)

install.packages("tabulizer", dependencies=TRUE)
tabulizer::extract_metadata("~/0.-miljoenennota.pdf")
#---------
$pages
[1] 196

$title
NULL

$author
[1] "Tweede Kamer der Staten-Generaal"

$subject
[1] ""

$keywords
[1] ""

$creator
[1] "XPP"

$producer
[1] "Acrobat Distiller Server 8.1.0 (Pentium Linux, Built: 2007-09-07)"

$created
[1] "Thu Sep 15 05:11:50 PDT 2016"

$modified
[1] "Thu Sep 15 05:34:06 PDT 2016"

$trapped
NULL