官網(wǎng):http://gearman.org/
跨多種環(huán)境部署 Gearman
http://www.ibm.com/developerworks/cn/opensource/os-gearman/index.html
利用開源的Gearman框架構(gòu)建分布式圖片處理平臺(tái)-張宴
http://blog.s135.com/dips/
監(jiān)控:
https://github.com/yugene/Gearman-Monitor
一、簡(jiǎn)介
Gearman是一個(gè)分發(fā)任務(wù)的程序架構(gòu),由三部分組成:
Gearman client:提供gearman client API給應(yīng)用程序調(diào)用。API可以使用C,PHP,PERL,MYSQL UDF等待呢個(gè)語言,它是請(qǐng)求的發(fā)起者。
Gearman job server:將客戶端的請(qǐng)求分發(fā)到各個(gè)gearman worker的調(diào)度者,相當(dāng)于中央控制器,但它不處理具體業(yè)務(wù)邏輯。
Gearman worker:提供gearman worker API給應(yīng)用程序調(diào)用,具體負(fù)責(zé)客戶端的請(qǐng)求,并將處理結(jié)果返回給客戶端。
Mogilefs的分布式文件系統(tǒng)的核心就是用gearman實(shí)現(xiàn)的。
這個(gè)軟件的應(yīng)用場(chǎng)景很多,比如視頻網(wǎng)站的視頻處理,分布式日志處理,電子郵件處理,文件同步處理,圖片處理等等,只要是可以放開,不影響體驗(yàn)和響應(yīng)的場(chǎng) 景,需要并行進(jìn)行大量計(jì)算和處理的程序都是可以的。Yahoo在60或更多的服務(wù)器上使用gearman每天處理600萬個(gè)作業(yè)。新聞聚合器digg構(gòu)建 了一個(gè)相同規(guī)模的gearman網(wǎng)絡(luò),每天可處理400000個(gè)作業(yè)。
Gearman不但可以做為任務(wù)分發(fā),還可以做為應(yīng)用方面的負(fù)載均衡。可以讓worker放在不同的一堆服務(wù)器上,也可以啟動(dòng)放在同一個(gè)cpu的多個(gè)核 上。比如,應(yīng)用視頻轉(zhuǎn)換程序,不希望web服務(wù)器來處理視頻格式轉(zhuǎn)換,這時(shí),可以在這一堆服務(wù)器上進(jìn)行任務(wù)分發(fā),在上面加載worker處理視頻格式,對(duì) 外的web服務(wù)器就不會(huì)被視頻轉(zhuǎn)換過程影響。而且擴(kuò)展方便,加一臺(tái)服務(wù)器到任務(wù)調(diào)度中心,注冊(cè)成worker即可,這時(shí)job server會(huì)在請(qǐng)求到來的時(shí)候,將請(qǐng)求發(fā)送給空閑的worker。還可以運(yùn)行多個(gè)job server,組成ha架構(gòu),如果一個(gè)job server當(dāng)?shù)袅耍琧lient和worker會(huì)自動(dòng)遷移到另一臺(tái)job server上。
二、安裝
[Job Server (gearmand) -- 172.16.1.183]
1.首先安裝libdrizzle
#yum install libdrizzle libdrizzle-devel
2.安裝gearman(兩種方法1.yum2.源碼包)。(c版的server)
1)yum安裝
#rpm -ivh http://dl.iuscommunity.org/pub/ius/stable/Redhat/6/x86_64/epel-release-6-5.noarch.rpm
#yum install -y gearmand
2)源碼包安裝
#cd /opt/build/
#wget https://launchpad.net/gearmand/trunk/0.34/+download/gearmand-0.34.tar.gz
#tar zxf gearmand-0.34.tar.gz
#cd gearmand-0.34
#./configure
#make && make install
3.啟動(dòng)gearman服務(wù)
1)yum安裝方式
#/etc/init.d/gearmand start
2)源碼包安裝方式
#/opt/build/gearmand-0.34/sbin/gearmand -d
#gearmand -vvv -u root
INFO Starting up
INFO Listening on :::4730 (6)
INFO Creating wakeup pipe
INFO Creating IO thread wakeup pipe
INFO Adding event for listening socket (6)
INFO Adding event for wakeup pipe
INFO Entering main event loop
worker&&client以php方式
[worker -- 172.16.1.180]
安裝gearmand如上所示
安裝 Gearman PHP extension
1.下載gearman-0.8.0.tgz并安裝
#cd /opt/build/
#wget http://pecl.php.net/get/gearman-0.8.0.tgz
# yum install -y libgearman-devel.x86_64
# yum install -y re2c
#tar zxf gearman-0.8.0.tgz
#cd gearman-0.8.0.tgz
#phpize
# ./configure
# make && make install
2.編輯php.ini配置文件加載相應(yīng)模塊并使之生效
# vim /etc/php.ini
extension = "gearman.so"
3.查看gearman.so模塊是否加載
# php --info | grep gearman
gearman
gearman support => enabled
libgearman version => 0.14
PWD => /opt/build/gearman-0.8.0
_SERVER["PWD"] => /opt/build/gearman-0.8.0
# php -m | grep gearman
gearman
4.啟動(dòng)job
gearmand -d
如果當(dāng)前用戶是 root 的話,則需要這樣操作:
gearmand -d -u root
缺省會(huì)使用 4730 端口,下面會(huì)用到。
注意:如果找不到 gearmand 命令的路徑,別忘了用 whereis gearmand 確認(rèn)
[client -- 172.16.1.181]
安裝如work同。如上所示。
三、測(cè)試:
[Job Server (gearmand) -- 172.16.1.183]
啟動(dòng)gearmand
以命令行工具來驗(yàn)證gearman的功能
啟動(dòng) Worker:gearman -h 172.16.1.183 -w -f wc -- wc -l &
運(yùn)行Client:gearman -h 172.16.1.183 -f wc < /etc/passwd
42
可以看到驗(yàn)證成功。
以php驗(yàn)證gearman的功能
編寫 Worker
worker.php 文件內(nèi)容如下:
<?php
$worker= new GearmanWorker();
$worker->addServer('172.16.1.183', 4730);
$worker->addFunction('reverse', 'my_reverse_function');
while ($worker->work());
function my_reverse_function($job) {
return strrev($job->workload());
}
?>
設(shè)置后臺(tái)運(yùn)行 work
php worker.php &
編寫 Client
client.php 文件內(nèi)容如下:
<?php
$client= new GearmanClient();
$client->addServer('172.16.1.183', 4730);
echo $client->do('reverse', 'Hello World!'), "\n";
?>
運(yùn)行 client
php client.php
輸出:!dlroW olleH
Q:
I've been trying to get Gearman compiled on CentOS 5.8 all afternoon. Unfortunately I am restricted to this version of CentOS by my CTO and how he has our entire network configured. I think it's simply because we don't have enough resources to upgrade our network... But anyways, the problem at hand.
I have searched through Server Fault, Stack Overflow, Google, and am unable to locate a working solution. What I have below is stuff I have pieced together from my searching.
Searches have told said to install the following via yum
:
yum -y install --enablerepo=remi boost141-devel libgearman-devel e2fsprogs-devel e2fsprogs gcc44 gcc-c++
To get the Boost headers working correctly I did this:
cp -f /usr/lib/boost141/* /usr/lib/ cp -f /usr/lib64/boost141/* /usr/lib64/ rm -f /usr/include/boost ln -s /usr/include/boost141/boost /usr/include/boost
With all of the dependancies installed and paths setup I then download and compile gearmand-1.1.2
just fine.
wget -O /tmp/gearmand-1.1.2.tar.gz https://launchpad.net/gearmand/1.2/1.1.2/+download/gearmand-1.1.2.tar.gz cd /tmp && tar zxvf gearmand-1.1.2.tar.gz ./configure && make -j8 && make install
That works correctly. So now I need to install the Gearman library for PHP. I have attempted through PECL and downloading the source directly, both result in the same error:
checking whether to enable gearman support... yes, shared not found configure: error: Please install libgearman
What I don't understand is I installed the libgearman-devel
package which also installed the core libgearman
. The installation installs libgearman-devel-0.14-3.el5.x86_64
, libgearman-devel-0.14-3.el5.i386
, libgearman-0.14-3.el5.x86_64
, and libgearman-0.14-3.el5.i386
.
Is it possible the package version is lower than what is required? I'm still poking around with this, but figured I'd throw this up to see if anyone has a solution while I continue to research a fix.
Thanks!
A:
This should do the trick:
export GEARMAN_LIB_DIR=/usr/include/libgearman
export GEARMAN_INC_DIR=/usr/include/libgearman
That should work, if not you'll have to do some minor edits to config.m4.
other:
http://gearman.org/gearman_php_extension
http://blog.csdn.net/aidenliu/article/details/7406390
http://www.php.net/manual/en/gearmanclient.dobackground.php
http://www.wenzizone.com/2012/09/27/how_to_fix_rpm_filedigests_payloadisxz_is_needed.html
http://www.2cto.com/os/201206/136785.html
http://blog.s135.com/dips
http://blog.csdn.net/hfahe/article/details/5519582
http://hi.baidu.com/sunjiujiu/item/4406281c952cf47a7b5f2594