kazu634
/
blog
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
blog/content/post/2009-09-26-00001227.md

138 lines
8.9 KiB
Markdown
Raw Normal View History

initial commit
2019-03-31 11:00:21 +00:00
---
title: Yahoo!ファイナンスのページから外国為替情報を取得し、DBに登録する
author: kazu634
date: 2009-09-26
url: /2009/09/26/_1335/
wordtwit_post_info:
- 'O:8:"stdClass":13:{s:6:"manual";b:0;s:11:"tweet_times";i:1;s:5:"delay";i:0;s:7:"enabled";i:1;s:10:"separation";s:2:"60";s:7:"version";s:3:"3.7";s:14:"tweet_template";b:0;s:6:"status";i:2;s:6:"result";a:0:{}s:13:"tweet_counter";i:2;s:13:"tweet_log_ids";a:1:{i:0;i:4797;}s:9:"hash_tags";a:0:{}s:8:"accounts";a:1:{i:0;s:7:"kazu634";}}'
categories:
- mysql
- Perl
---
<div class="section">
<p>
日本円が、各国の通貨に対してどの程度の価値を持っているかを自動で取得できるようにしてみました。
</p>
<h4>
MySQLのテーブルの準備
</h4>
<p>
今回はMySQLで次のようなデータベースとテーブルを準備しました。
</p>
<pre class="syntax-highlight">
mysql&#62; use money;
Database changed
mysql&#62; desc JPY;
+-------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| date | datetime | YES | | NULL | |
| USD | float(10,7) | YES | | NULL | |
| AUD | float(10,7) | YES | | NULL | |
| GBP | float(10,7) | YES | | NULL | |
| CAD | float(10,7) | YES | | NULL | |
| CHF | float(10,7) | YES | | NULL | |
| EUR | float(10,7) | YES | | NULL | |
+-------+-------------+------+-----+---------+----------------+
8 rows in set (0.00 sec)
mysql&#62;
</pre>
<h4>
perlのスクリプト
</h4>
<p>
今回作成したのは、次のようなスクリプトです。
</p>
<pre class="syntax-highlight">
<span class="synComment"># === Libraries ===</span>
<span class="synStatement">use strict</span>;
<span class="synStatement">use warnings</span>;
<span class="synStatement">use </span>Perl6::Say;
<span class="synStatement">use </span>YAML;
<span class="synStatement">use </span>URI;
<span class="synStatement">use </span>Web::Scraper;
<span class="synStatement">use </span>Encode;
<span class="synStatement">use utf8</span>;
<span class="synStatement">use </span>DBI;
<span class="synStatement">use </span>POSIX <span class="synConstant">'strftime'</span>;
<span class="synComment"># === Main part ===</span>
<span class="synStatement">my</span> <span class="synIdentifier">$scraper</span> = scraper {
process <span class="synConstant">'//table[@class=&#34;yjS&#34;]/tr[2]'</span>, <span class="synConstant">'JPY'</span> =&#62; scraper {
process <span class="synConstant">'//td[2]'</span>, <span class="synConstant">'USD'</span> =&#62; <span class="synConstant">'TEXT'</span>;
process <span class="synConstant">'//td[3]'</span>, <span class="synConstant">'AUD'</span> =&#62; <span class="synConstant">'TEXT'</span>;
process <span class="synConstant">'//td[4]'</span>, <span class="synConstant">'GBP'</span> =&#62; <span class="synConstant">'TEXT'</span>;
process <span class="synConstant">'//td[5]'</span>, <span class="synConstant">'CAD'</span> =&#62; <span class="synConstant">'TEXT'</span>;
process <span class="synConstant">'//td[6]'</span>, <span class="synConstant">'CHF'</span> =&#62; <span class="synConstant">'TEXT'</span>;
process <span class="synConstant">'//td[7]'</span>, <span class="synConstant">'EUR'</span> =&#62; <span class="synConstant">'TEXT'</span>;
};
};
<span class="synStatement">my</span> <span class="synIdentifier">$result</span> = <span class="synIdentifier">$scraper</span>-&#62;scrape( URI-&#62;<span class="synStatement">new</span>(<span class="synConstant">&#34;http://quote.yahoo.co.jp/m3&#34;</span>) );
<span class="synComment"># print encode( 'utf8', YAML::Dump($result) );</span>
<span class="synComment"># データベースへの接続</span>
<span class="synStatement">my</span> <span class="synIdentifier">$dbh</span> =
DBI-&#62;<span class="synStatement">connect</span>( <span class="synConstant">'dbi:mysql:dbname=money'</span>, <span class="synConstant">'root'</span>, <span class="synConstant">'musashi'</span>,
{ <span class="synConstant">RaiseError </span>=&#62; <span class="synConstant">1</span>, <span class="synConstant">AutoCommit </span>=&#62; <span class="synConstant"></span> } );
<span class="synComment"># ステートメントハンドラの作成</span>
<span class="synComment"># my $sth = $dbh-&#62;prepare(&#34;SELECT address FROM renoir WHERE address LIKE ?;&#34;);</span>
<span class="synStatement">my</span> <span class="synIdentifier">$sth</span> = <span class="synIdentifier">$dbh</span>-&#62;prepare(
<span class="synConstant">&#34;INSERT INTO JPY(date, USD, AUD, GBP, CAD, CHF, EUR) VALUES (?, ?, ?, ?, ?, ?, ?);&#34;</span>
);
<span class="synComment"># MySQLの日付時刻型( datetime型 )</span>
<span class="synStatement">my</span> <span class="synIdentifier">$current_datetime</span> = strftime( <span class="synConstant">&#34;</span><span class="synIdentifier">%Y</span><span class="synConstant">-</span><span class="synIdentifier">%m</span><span class="synConstant">-</span><span class="synIdentifier">%d</span><span class="synConstant"> </span><span class="synIdentifier">%H</span><span class="synConstant">:</span><span class="synIdentifier">%M</span><span class="synConstant">:</span><span class="synIdentifier">%S</span><span class="synConstant">&#34;</span>, <span class="synStatement">localtime</span> );
<span class="synComment"># ステートメントハンドラの実行</span>
<span class="synComment"># もしプレースホルダを用いていれば、引数を指定する</span>
<span class="synComment"># 例: $sth-&#62;execute(&#34;$query%&#34;);</span>
<span class="synIdentifier">$sth</span>-&#62;execute(
<span class="synIdentifier">$current_datetime</span>, <span class="synIdentifier">$result</span>-&#62;{<span class="synConstant">'JPY'</span>}-&#62;{<span class="synConstant">'USD'</span>},
<span class="synIdentifier">$result</span>-&#62;{<span class="synConstant">'JPY'</span>}-&#62;{<span class="synConstant">'AUD'</span>}, <span class="synIdentifier">$result</span>-&#62;{<span class="synConstant">'JPY'</span>}-&#62;{<span class="synConstant">'GBP'</span>},
<span class="synIdentifier">$result</span>-&#62;{<span class="synConstant">'JPY'</span>}-&#62;{<span class="synConstant">'CAD'</span>}, <span class="synIdentifier">$result</span>-&#62;{<span class="synConstant">'JPY'</span>}-&#62;{<span class="synConstant">'CHF'</span>},
<span class="synIdentifier">$result</span>-&#62;{<span class="synConstant">'JPY'</span>}-&#62;{<span class="synConstant">'EUR'</span>}
);
<span class="synComment"># ステートメントハンドラの解放</span>
<span class="synIdentifier">$sth</span>-&#62;finish;
<span class="synComment"># データベースハンドラの解放</span>
<span class="synIdentifier">$dbh</span>-&#62;disconnect;
</pre>
<h4>
実行結果
</h4>
<pre class="syntax-highlight">
kazu634@srv634% perl 200909252353_money.pl ~/work/tmp_perl/scrap <span class="synStatement">[</span><span class="synConstant">6573</span><span class="synStatement">]</span>
---
JPY:
AUD: <span class="synConstant">77</span>.<span class="synConstant">790160</span>
CAD: <span class="synConstant">82</span>.<span class="synConstant">107192</span>
CHF: <span class="synConstant">87</span>.<span class="synConstant">145080</span>
EUR: <span class="synConstant">131</span>.<span class="synConstant">633856</span>
GBP: <span class="synConstant">142</span>.<span class="synConstant">961824</span>
USD: <span class="synConstant">89</span>.<span class="synConstant">620000</span>
</pre>
<p>
MySQLで確認してみます:
</p>
<pre class="syntax-highlight">
mysql&#62; <span class="synStatement">SELECT</span> * <span class="synSpecial">FROM</span> JPY\G
*************************** <span class="synConstant">1</span>. <span class="synSpecial">row</span> ***************************
id: <span class="synConstant">1</span>
<span class="synType">date</span>: <span class="synConstant">2009-09-26</span> <span class="synConstant">11</span>:<span class="synConstant">31</span>:<span class="synConstant">16</span>
USD: <span class="synConstant">89</span>.<span class="synConstant">6200027</span>
AUD: <span class="synConstant">77</span>.<span class="synConstant">7901611</span>
GBP: <span class="synConstant">142</span>.<span class="synConstant">9618225</span>
CAD: <span class="synConstant">82</span>.<span class="synConstant">1071930</span>
CHF: <span class="synConstant">87</span>.<span class="synConstant">1450806</span>
EUR: <span class="synConstant">131</span>.<span class="synConstant">6338501</span>
<span class="synConstant">1</span> <span class="synSpecial">row</span> <span class="synStatement">in</span> <span class="synStatement">set</span> (<span class="synConstant"></span>.<span class="synConstant">00</span> sec)
</pre>
</div>