代码之家  ›  专栏  ›  技术社区  ›  Adam Outler

在bash中更快地处理文本数据

performance text string bash
  •  3
  • Adam Outler  · 技术社区  · 15 年前

    我有个速度问题。我有一个bash脚本,它解析来自thetvdb.com的信息。它下载了近40000行数据,然后将其减少到大约5000行数据,这些数据被写入硬盘。然后它读取文件并将其解析为几个文件,这些文件稍后用作查找表。它基本上是把它在每一集之前看到的所有信息都写进一个特定的文件,然后重设为下一集。

    它必须在“/epision”标签上同步,因为在“epision”标签之外有一个“firstaired”标签。这样可以确保数据是按顺序绘制的,而不是根据与某个事件相关的每个单独标记来绘制。

    这是有问题的代码。 

      if [ -f "$mythicalLibrarian/$NewShowName/$NewShowName.xml" ]; then
   Ename=""
   actualEname=""
   FAired=""
   SeasonNr=""
   EpisodeNr=""
    recordNumber=0

    echo "Parsing Downloaded information: $NewShowName.xml "
    while read line
   do

     if [[ $line == \<\/Episode\> ]]; then
      (( ++recordNumber ))
      echo -ne "Building Record:$recordNumber ${actualEname:0:20}            \r" 1>&2 
     echo "$actualEname" >> "$mythicalLibrarian/$NewShowName/$NewShowName.actualEname.txt"&
      Ename=`echo "$actualEname" |sed 's/;.*//'`
     echo "$Ename" >> "$mythicalLibrarian/$NewShowName/$NewShowName.Ename.txt"&
     echo "$FAired" >> "$mythicalLibrarian/$NewShowName/$NewShowName.FAired.txt"&
     echo "$SeasonNr" >> "$mythicalLibrarian/$NewShowName/$NewShowName.S.txt"&
     echo "$EpisodeNr" >> "$mythicalLibrarian/$NewShowName/$NewShowName.E.txt"&
     Ename=""
     actualEname=""
     FAired=""
     SeasonNr=""
     EpisodeNr=""

#Get actual show name 
     elif [[ $line == \<EpisodeName\>* ]]; then
      actualEname=`echo "$line" | sed -e s/'<\/EpisodeName>'// -e s/'<EpisodeName>'// -e s/'\&amp\;'/'\&'/ -e s/'\&quot\;'/'\"'/ -e s/'\&amp\;'/'\&'/ -e s/'\&ndash\;'/'-'/ -e s/'\&lt\;'/'\<'/ -e 's/'\&gt\;'/'\>'/' |tr -d '|\?\*\<\"\:\>\+\\\[\]\/'`


#Get OriginalAirDate
    elif [[ $line == \<FirstAired\>* ]]; then
      FAired=`echo "$line" | sed -e s/'<FirstAired>'//g -e s/'<\/FirstAired>'//g`

#Get Season number
     elif [[ $line == \<SeasonNumber\>* ]]; then
      SeasonNr=`echo "$line" |sed -e s/'<SeasonNumber>'// -e s/'<\/SeasonNumber>'//`

#Get Episode number
    elif [[ $line == \<EpisodeNumber\>* ]]; then
      EpisodeNr=`echo "$line" |sed -e 's/<EpisodeNumber>//' -e 's/<\/EpisodeNumber>//'`

    fi
   done < "$mythicalLibrarian/$NewShowName/$NewShowName.xml"


   chmod 777 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".actualEname.txt
   chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".Ename.txt
   chmod 666 "$mythicalLibrarian/$NewShowName/$NewShowName".FAired.txt
   chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".S.txt
   chmod 666 "$mythicalLibrarian/$NewShowName/$NewShowName".E.txt
    GotNewInformation=1


  elif [ ! -f "$mythicalLibrarian/$NewShowName/$NewShowName.xml" ]; then
   echo "COULD NOT DOWNLOAD:www.thetvdb.com/api/$APIkey/series/$SeriesID/all/$Language.xml">>"$mythicalLibrarian"/output.log
  fi

    这是它正在处理的一些数据 

    <?xml version="1.0" encoding="UTF-8" ?>
<Data><Series>
  <Actors>|Fred Rogers|Adair Roth|Bert Lloyd|Bud Alder|Carol Saunders|Carole Switala|Deborah Neal Stampo|Don Brockett|Elsie Neal|Emilie Jacobson|Fred Michael|John Reardon|Jos|Judy Rubin|Keith David|Lenny Meledandri|Michael Horton|Robert Trow|Yoshi Ito|</Actors>
  <Airs_DayOfWeek></Airs_DayOfWeek>
  <Airs_Time></Airs_Time>
  <ContentRating></ContentRating>
  <FirstAired>1968-02-01</FirstAired>
  <Genre>|Children|</Genre>
  <Network>PBS</Network>
  <NetworkID></NetworkID>
  <Overview>&quot;In a little toy neighborhood, a tiny trolley rolls past a house at the end of a street.

  <Runtime>30</Runtime>
  <SeriesID>6843</SeriesID>
  <SeriesName>Mister Rogers' Neighborhood</SeriesName>
  <Status>Ended</Status>
  <added></added>
  <addedBy></addedBy>
  <banner>graphical/77750-g.jpg</banner>
  <fanart>fanart/original/77750-1.jpg</fanart>
  <poster></poster>
  <zap2it_id>SH002930</zap2it_id>
</Series>
<Episode>
  <EpisodeName>Change (1)</EpisodeName>
  <EpisodeNumber>1</EpisodeNumber>
  <FirstAired>1968-02-19</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Change (2)</EpisodeName>
  <EpisodeNumber>2</EpisodeNumber>
  <FirstAired>1968-02-20</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Change (3)</EpisodeName>
  <EpisodeNumber>3</EpisodeNumber>
  <FirstAired>1968-02-21</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Change (4)</EpisodeName>
  <EpisodeNumber>4</EpisodeNumber>
  <FirstAired>1968-02-22</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Change (5)</EpisodeName>
  <EpisodeNumber>5</EpisodeNumber>
  <FirstAired>1968-02-23</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 6</EpisodeName>
  <EpisodeNumber>6</EpisodeNumber>
  <FirstAired>1968-02-26</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 7</EpisodeName>
  <EpisodeNumber>7</EpisodeNumber>
  <FirstAired>1968-02-27</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 8</EpisodeName>
  <EpisodeNumber>8</EpisodeNumber>
  <FirstAired>1968-02-28</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 9</EpisodeName>
  <EpisodeNumber>9</EpisodeNumber>
  <FirstAired>1968-02-29</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 10</EpisodeName>
  <EpisodeNumber>10</EpisodeNumber>
  <FirstAired>1968-03-01</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 11</EpisodeName>
  <EpisodeNumber>11</EpisodeNumber>
  <FirstAired>1968-03-04</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 12</EpisodeName>
  <EpisodeNumber>12</EpisodeNumber>
  <FirstAired>1968-03-05</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 13</EpisodeName>
  <EpisodeNumber>13</EpisodeNumber>
  <FirstAired>1968-03-06</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 14</EpisodeName>
  <EpisodeNumber>14</EpisodeNumber>
  <FirstAired>1968-03-07</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 15</EpisodeName>
  <EpisodeNumber>15</EpisodeNumber>
  <FirstAired>1968-03-08</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Welcome Donkey Hodie (1)</EpisodeName>
  <EpisodeNumber>16</EpisodeNumber>
  <FirstAired>1968-03-11</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Welcome Donkey Hodie (2)</EpisodeName>
  <EpisodeNumber>17</EpisodeNumber>
  <FirstAired>1968-03-12</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Welcome Donkey Hodie (3)</EpisodeName>
  <EpisodeNumber>18</EpisodeNumber>
  <FirstAired>1968-03-13</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Welcome Donkey Hodie (4)</EpisodeName>
  <EpisodeNumber>19</EpisodeNumber>
  <FirstAired>1968-03-14</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Welcome Donkey Hodie (5)</EpisodeName>
  <EpisodeNumber>20</EpisodeNumber>
  <FirstAired>1968-03-15</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 21</EpisodeName>
  <EpisodeNumber>21</EpisodeNumber>
  <FirstAired>1968-03-18</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 22</EpisodeName>
  <EpisodeNumber>22</EpisodeNumber>
  <FirstAired>1968-03-19</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 23</EpisodeName>
  <EpisodeNumber>23</EpisodeNumber>
  <FirstAired>1968-03-20</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 24</EpisodeName>
  <EpisodeNumber>24</EpisodeNumber>
  <FirstAired>1968-03-21</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 25</EpisodeName>
  <EpisodeNumber>25</EpisodeNumber>
  <FirstAired>1968-03-22</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 26</EpisodeName>
  <EpisodeNumber>26</EpisodeNumber>
  <FirstAired>1968-03-25</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 27</EpisodeName>
  <EpisodeNumber>27</EpisodeNumber>
  <FirstAired>1968-03-26</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 28</EpisodeName>
  <EpisodeNumber>28</EpisodeNumber>
  <FirstAired>1968-03-27</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 29</EpisodeName>
  <EpisodeNumber>29</EpisodeNumber>
  <FirstAired>1968-03-28</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 30</EpisodeName>
  <EpisodeNumber>30</EpisodeNumber>
  <FirstAired>1968-03-29</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Red Monster (1)</EpisodeName>
  <EpisodeNumber>31</EpisodeNumber>
  <FirstAired>1968-04-01</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Red Monster (2)</EpisodeName>
  <EpisodeNumber>32</EpisodeNumber>
  <FirstAired>1968-04-02</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Red Monster (3)</EpisodeName>
  <EpisodeNumber>33</EpisodeNumber>
  <FirstAired>1968-04-03</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Red Monster (4)</EpisodeName>
  <EpisodeNumber>34</EpisodeNumber>
  <FirstAired>1968-04-04</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Red Monster (5)</EpisodeName>
  <EpisodeNumber>35</EpisodeNumber>
  <FirstAired>1968-04-05</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 36</EpisodeName>
  <EpisodeNumber>36</EpisodeNumber>
  <FirstAired>1968-04-08</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 37</EpisodeName>
  <EpisodeNumber>37</EpisodeNumber>
  <FirstAired>1968-04-09</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 38</EpisodeName>
  <EpisodeNumber>38</EpisodeNumber>
  <FirstAired>1968-04-10</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 39</EpisodeName>
  <EpisodeNumber>39</EpisodeNumber>
  <FirstAired>1968-04-11</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 40</EpisodeName>
  <EpisodeNumber>40</EpisodeNumber>
  <FirstAired>1968-04-12</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 41</EpisodeName>
  <EpisodeNumber>41</EpisodeNumber>
  <FirstAired>1968-04-15</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 42</EpisodeName>
  <EpisodeNumber>42</EpisodeNumber>
  <FirstAired>1968-04-16</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 43</EpisodeName>
  <EpisodeNumber>43</EpisodeNumber>
  <FirstAired>1968-04-17</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 44</EpisodeName>
  <EpisodeNumber>44</EpisodeNumber>
  <FirstAired>1968-04-18</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 45</EpisodeName>
  <EpisodeNumber>45</EpisodeNumber>
  <FirstAired>1968-04-19</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 46</EpisodeName>
  <EpisodeNumber>46</EpisodeNumber>
  <FirstAired>1968-04-22</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 47</EpisodeName>
  <EpisodeNumber>47</EpisodeNumber>
  <FirstAired>1968-04-23</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 48</EpisodeName>
  <EpisodeNumber>48</EpisodeNumber>
  <FirstAired>1968-04-24</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 49</EpisodeName>
  <EpisodeNumber>49</EpisodeNumber>
  <FirstAired>1968-04-25</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 50</EpisodeName>
  <EpisodeNumber>50</EpisodeNumber>
  <FirstAired>1968-04-26</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 51</EpisodeName>
  <EpisodeNumber>51</EpisodeNumber>
  <FirstAired>1968-04-29</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 52</EpisodeName>
  <EpisodeNumber>52</EpisodeNumber>
  <FirstAired>1968-04-30</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 53</EpisodeName>
  <EpisodeNumber>53</EpisodeNumber>
  <FirstAired>1968-05-01</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 54</EpisodeName>
  <EpisodeNumber>54</EpisodeNumber>
  <FirstAired>1968-05-02</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 55</EpisodeName>
  <EpisodeNumber>55</EpisodeNumber>
  <FirstAired>1968-05-03</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 56</EpisodeName>
  <EpisodeNumber>56</EpisodeNumber>
  <FirstAired>1968-05-06</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 57</EpisodeName>
  <EpisodeNumber>57</EpisodeNumber>
  <FirstAired>1968-05-07</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 58</EpisodeName>
  <EpisodeNumber>58</EpisodeNumber>
  <FirstAired>1968-05-08</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 59</EpisodeName>
  <EpisodeNumber>59</EpisodeNumber>
  <FirstAired>1968-05-09</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 60</EpisodeName>
  <EpisodeNumber>60</EpisodeNumber>
  <FirstAired>1968-05-10</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 61</EpisodeName>
  <EpisodeNumber>61</EpisodeNumber>
  <FirstAired>1968-05-13</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 62</EpisodeName>
  <EpisodeNumber>62</EpisodeNumber>
  <FirstAired>1968-05-14</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 63</EpisodeName>
  <EpisodeNumber>63</EpisodeNumber>
  <FirstAired>1968-05-15</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 64</EpisodeName>
  <EpisodeNumber>64</EpisodeNumber>
  <FirstAired>1968-05-16</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 65</EpisodeName>
  <EpisodeNumber>65</EpisodeNumber>
  <FirstAired>1968-05-17</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 66</EpisodeName>
  <EpisodeNumber>66</EpisodeNumber>
  <FirstAired>1968-05-20</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 67</EpisodeName>
  <EpisodeNumber>67</EpisodeNumber>
  <FirstAired>1968-05-21</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 68</EpisodeName>
  <EpisodeNumber>68</EpisodeNumber>
  <FirstAired>1968-05-22</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 69</EpisodeName>
  <EpisodeNumber>69</EpisodeNumber>
  <FirstAired>1968-05-23</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 70</EpisodeName>
  <EpisodeNumber>70</EpisodeNumber>
  <FirstAired>1968-05-24</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 71</EpisodeName>
  <EpisodeNumber>71</EpisodeNumber>
  <FirstAired>1968-05-27</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Show 72</EpisodeName>
  <EpisodeNumber>72</EpisodeNumber>
  <FirstAired>1968-05-28</FirstAired>
  <SeasonNumber>1</SeasonNumber>
</Episode>

    问题是,在i7处理器上,这需要14.5秒。在我的Media Center上大约慢10倍。我尝试使用一个case语句,它在快速处理器上需要15秒。

    我想知道如何加快这个过程。对于bash来说,这个速度似乎慢得离谱,bash应该围绕数据操作和文件操作进行设计。

    6 回复  |  直到 15 年前
        1
  •  4
  •   Dennis Williamson    15 年前

    & echo 

    $ time { for i in {1..1000}; do echo "hello"& done >/dev/null; } | cat

real    0m10.357s
user    0m2.764s
sys     0m15.441s

    cat 

    $ time { for i in {1..1000}; do echo "hello"; done >/dev/null; }

real    0m0.152s
user    0m0.132s
sys     0m0.020s


    elif [[ $line == \<EpisodeName\>* ]]; then
    actualEname=${line//<\/EpisodeName>/}
    actualEname=${actualEname//<EpisodeName>/}
    actualEname=${actualEname//&amp;/&}
    actualEname=${actualEname//&ndash;/-}
    for string in '|' '&lt;' '&gt;' '&quot;' '?' '*' '<' '>' ':' '"' '+' '\' '[' ']' '/'
    do
        actualEname=${actualEname//$string}
    done

    &amp; g 

    $ time { for i in {1..100}; do
    line='<EpisodeName>&lt;foo&amp;bar&ndash;baz&gt;Season&ndash;3&ndash;&quot;quux&quot;?*<>:"+\[]/</EpisodeName>'
    actualEname=$(echo "$line" | sed -e 's/<\/EpisodeName>//' -e 's/<EpisodeName>//' -e 's/&amp;/\&/g' -e 's/&quot;/"/g' -e 's/&ndash;/-/g' -e 's/&lt;/</g' -e 's/&gt;/>/g' |tr -d '|?*<":>+\\[]/')
done; }

real    0m7.779s
user    0m3.164s
sys     0m5.436s


    $ time { for i in {1..100}; do
    line='<EpisodeName>&lt;foo&amp;bar&ndash;baz&gt;Season&ndash;3&ndash;&quot;quux&quot;?*<>:"+\[]/</EpisodeName>
    actualEname=${line//<\/EpisodeName>/}
    actualEname=${actualEname//<EpisodeName>/}
    actualEname=${actualEname//&amp;/&}
    actualEname=${actualEname//&ndash;/-}
    for string in '|' '&lt;' '&gt;' '&quot;' '\?' '\*' '<' '>' ':' '"' '+' '\\' '[' ']' '\/'
    do
        actualEname=${actualEname//$string}
    done
done; }

real    0m5.403s
user    0m2.492s
sys     0m2.960s
        2
  •  1
  •   Ignacio Vazquez-Abrams    15 年前

    XMLStarlet

        3
  •  0
  •   Orbling    15 年前

    sed tr

    XSLT

    就我个人而言,我会用Perl做这类事情。

        4
  •  0
  •   msw    15 年前

    bash应该围绕数据操作和文件操作进行设计。

    bash是为交互式命令处理和通过管道将程序链接在一起而设计的。重型数据处理并不是我所知道的任何一个*sh的设计空间。

    对于问题空间,python或perl是更好的选择。

        5
  •  0
  •   Dennis Williamson    15 年前

    我刚试过这个: 

            echo "Parsing Downloaded information: $NewShowName.xml "
        while read line
        do




            if [[ $line == \<\/Episode\> ]]; then
                (( ++recordNumber ))
                echo -ne "Building Record:$recordNumber ${actualEname:0:20}            \r" 1>&2 
                echo "$EpisodeName" >> "$mythicalLibrarian/$NewShowName/$NewShowName.actualEname.txt"&
                Ename=`echo "$actualEname" |sed 's/;.*//'`
                echo "$EpisodeName" >> "$mythicalLibrarian/$NewShowName/$NewShowName.Ename.txt"&
                echo "$FirstAired" >> "$mythicalLibrarian/$NewShowName/$NewShowName.FAired.txt"&
                echo "$SeasonNumber" >> "$mythicalLibrarian/$NewShowName/$NewShowName.S.txt"&
                echo "$EpisodeNumber" >> "$mythicalLibrarian/$NewShowName/$NewShowName.E.txt"&
                EpisodeName=""
                actualEname=""
                FirstAired=""
                SeasonNumber=""
                EpisodeNumber=""
            else 
                var=`echo $line |tr '<>' ' '|awk '{print $1}'`

                value=`echo "$line"|sed -e s/'<'"$var"'>'// -e s/'<\/'"$var"'>'// -e s/'\&amp\;'/'\&'/ -e s/'\&quot\;'/'\"'/ -e s/'\&amp\;'/'\&'/ -e s/'\&ndash\;'/'-'/ -e s/'\&lt\;'/'\<'/ -e 's/'\&gt\;'/'\>'/' |tr -d '|\?\*\<\"\:\>\+\\\[\]\/'`
                eval $var="'$value'"
            fi

    在更快的处理器上用了43秒

        6
  •  0
  •   Adam Outler    15 年前

    天哪,丹尼斯威廉姆森,它在不到1/2秒的时间内解析。它只是在屏幕上闪烁。它过去需要15秒,但现在它太快了,我甚至都说不出它在发生。

    这些是丹尼斯威廉姆森建议的改变。我只是把它贴在这里。 

                echo "Parsing Downloaded information: $NewShowName.xml "
            while read line
            do

                if [[ $line == \<\/Episode\> ]]; then
                    (( ++recordNumber ))
                    echo -ne "Building Record:$recordNumber ${actualEname:0:20}           \r" 1>&2 
                    echo "$actualEname" >> "$mythicalLibrarian/$NewShowName/$NewShowName.actualEname.txt"

                    echo "$Ename" >> "$mythicalLibrarian/$NewShowName/$NewShowName.Ename.txt"
                    echo "$FAired" >> "$mythicalLibrarian/$NewShowName/$NewShowName.FAired.txt"
                    echo "$SeasonNr" >> "$mythicalLibrarian/$NewShowName/$NewShowName.S.txt"
                    echo "$EpisodeNr" >> "$mythicalLibrarian/$NewShowName/$NewShowName.E.txt"
                    Ename=""
                    actualEname=""
                    FAired=""
                    SeasonNr=""
                    EpisodeNr=""

#Get actual show name   
                elif [[ $line == \<EpisodeName\>* ]]; then
                    line=${line/<\/EpisodeName>/}
                    line=${line/<EpisodeName>/}
                    line=${line/&lt;}
                    line=${line/&gt;/} 
                    line=${line/&quot;/} 
                    line=${line/&amp;/&}
                    line=${line/\|/}
                    line=${line/\?/}
                    line=${line/\*/}
                    line=${line/\:/}
                    line=${line/\+/}
                    line=${line/\\/}
                    line=${line/\//}
                    line=${line/\[/}
                    line=${line/\]/}
                    line=${line/\'/}
                    line=${line/\"/}
                    actualEname=${line/&ndash;/-}
                    Ename=${actualEname/;*/}

#Get OriginalAirDate
                elif [[ $line == \<FirstAired\>* ]]; then
                    line=${line/<\/FirstAired>/}
                    line=${line/<FirstAired>/}
                    FAired=$line

#Get Season number
                elif [[ $line == \<SeasonNumber\>* ]]; then
                    line=${line/<\/SeasonNumber>/}
                    line=${line/<SeasonNumber>/}
                    SeasonNr=$line

#Get Episode number
                elif [[ $line == \<EpisodeNumber\>* ]]; then
                    line=${line/<\/EpisodeNumber>/}
                    line=${line/<EpisodeNumber>/}
                    EpisodeNr=$line
                fi
            done < "$mythicalLibrarian/$NewShowName/$NewShowName.xml"


            chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".actualEname.txt
            chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".Ename.txt
            chmod 666 "$mythicalLibrarian/$NewShowName/$NewShowName".FAired.txt
            chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".S.txt
            chmod 666 "$mythicalLibrarian/$NewShowName/$NewShowName".E.txt
            GotNewInformation=1
    推荐文章
    Sweepy Dodo  ·  JSON lite的格式化
    1 年前
    Merlin Nestler  ·  为什么索引中的列顺序在这种情况下会产生如此大的影响?
    1 年前
    giantjenga  ·  优化整数向量到二进制向量的转换
    1 年前
    Adriaantje  ·  Powershell版本的cut-d在处理大文件时速度很慢,我是否错过了快速的方法?
    1 年前
    Costinean Sebastian  ·  如何优化使用for循环处理大型数组的MATLAB脚本的执行时间?"
    1 年前
    Zegarek  ·  Postgresql递归查询未提供预期结果
    1 年前
    Joe  ·  为什么这两个查询之间的性能存在如此大的差异?
    2 年前
    tic-toc-choc  ·  在`dplyr中高效使用列表进行过滤`
    2 年前
    Mohan  ·  是否有一种更快的方法来编写代码,从1:N中提取许多随机样本?
    2 年前
    user2980746  ·  在C#字典中键入xyz对的最有效方法是什么?
    2 年前
    关于   移动版
    代码之家 - 一站式码农服务社区
    沪ICP备11025650号