Ever tried , Ever failed !
No matter ,Try again ! Fail again , Fail better !

编译安装Azkaban3.43

1.下载Azkaban源码,下载地址如下:

wget https://codeload.github.com/azkaban/azkaban/tar.gz/3.43.0

2.解压

tar –zxvf 3.43.0

3.解压后的目录为azkaban-3.43.0

cd azkaban-3.43.0

4.编译Azkaban,如下命令进行编译:

./gradlew build

有点慢,我第一次编译失败了,重新编译一遍就成功了

5.可以用以下命令查看编译好的安装包

ls */build/distributions/*

6.将编译成功的azkaban-*.tar.gz压缩包拷贝至/home/hadoop/azkaban目录下

mkdir -p /home/hadoop/azkaban
scp */build/distributions/*.tar.gz /home/hadoop/azkaban

7.将/home/hadoop/azkaban目录下的所有压缩包解压

tar -zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz 
tar -zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz 
tar -zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz
tar -zxvf azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz 
tar -zxvf azkaban-hadoop-security-plugin-0.1.0-SNAPSHOT.tar.gz 

8.重命名

mv azkaban-db-0.1.0-SNAPSHOT azkaban-db
mv azkaban-exec-server-0.1.0-SNAPSHOT azkaban-exec-server
mv azkaban-hadoop-security-plugin-0.1.0-SNAPSHOT azkaban-hadoop-security-plugin
mv azkaban-solo-server-0.1.0-SNAPSHOT azkaban-solo-server
mv azkaban-web-server-0.1.0-SNAPSHOT azkaban-web-server

安装包可以删除,看你自己

9.配置数据库

进入到数据库:创建Azkaban库及访问数据库的用户

# 创建数据库
create database azkaban default character set utf8;
# 添加权限
CREATE USER 'azkaban'@'%' IDENTIFIED BY 'password';
# 设置权限
GRANT ALL PRIVILEGES ON azkaban.* TO 'azkaban'@'%';
# 设置生效
FLUSH PRIVILEGES;

10.导入数据表,将/home/hadoop/azkaban/azkaban-db/create-all-sql-0.1.0-SNAPSHOT.sql导入数据库

# 选择表
use azkaban;
# 导入表
source /home/hadoop/azkaban/azkaban-db/create-all-sql-0.1.0-SNAPSHOT.sql
# 查看是否导入成功
show tables;

11.因为azkaban-web-serverazkaban-exec-server文件夹中没有confplugins文件夹,所以需要从azkaban-solo-server中拷贝一份

 scp -r azkaban-solo-server/conf/ azkaban-solo-server/plugins/ azkaban-web-server/
 scp -r azkaban-solo-server/conf/ azkaban-solo-server/plugins/ azkaban-exec-server/

12.分别在azkaban-web-server/confazkaban-exec-server/conf 中新建log4j.properties

vim log4j.properties
log4j.rootLogger=INFO,C
log4j.appender.C=org.apache.log4j.ConsoleAppender
log4j.appender.C.Target=System.err
log4j.appender.C.layout=org.apache.log4j.PatternLayout
log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

13.创建密钥文件:

keytool -keystore keystore -alias jetty -genkey -keyalg RSA

在当前目录生成一个 keystore 文件。将 keystore 拷贝到 Azkaban Web Server 目录中。

13.配置azkaban-web-server

vim /home/hadoop/azkaban/azkaban-web-server/conf/azkaban.properties

#zkaban Personalization Settings
azkaban.name=CDH_Azkaban
azkaban.label=My CDH Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml
# Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects
database.type=mysql
mysql.port=3306
mysql.host=hadoop000
mysql.database=azkaban
mysql.user=azkaban
mysql.password=password
mysql.numconnections=100
# Velocity dev mode
velocity.dev.mode=false
# Azkaban Jetty server properties.
jetty.use.ssl=true
jetty.maxThreads=25
jetty.port=8081
# Azkaban Executor settings
executor.port=12321
# mail settings
mail.sender=
mail.host=
# User facing web server configurations used to construct the user facing server URLs. They are useful when there
is a reverse proxy between Azkaban web servers and users.
# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
# when this parameters set then these parameters are used to generate email links.
# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
# azkaban.webserver.external_hostname=myazkabanhost.com
# azkaban.webserver.external_ssl_port=443
# azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
# Azkaban plugin settings
viewer.plugin.dir=plugins/jobtype
jetty.keystore=keystore
jetty.password=123456
jetty.keypassword=123456
jetty.truststore=keystore
jetty.trustpassword=123456

14.配置azkaban-exec-server

vim /home/hadoop/azkaban/azkaban-exec-server/conf/azkaban.properties

#zkaban Personalization Settings
default.timezone.id=Asia/Shanghai
# Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects
database.type=mysql
mysql.port=3306
mysql.host=hadoop000
mysql.database=azkaban
mysql.user=azkaban
mysql.password=123456
mysql.numconnections=100
# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=plugins/jobtypes

15.增加管理员用户

vim /home/hadoop/azkaban/azkaban-web-server/conf/azkaban-users.xml

默认用户名:azkaban 密码:azkaban

如果想增加管理员用户,在azkaban-users.xml添加用户:

<user password="admin" roles="admin,metrics" username="admin"/>
  • azkaban-users.xml
<azkaban-users>
  <user groups="azkaban" password="azkaban" roles="admin" username="azkaban"/>
  <user password="metrics" roles="metrics" username="metrics"/>

  <role name="admin" permissions="ADMIN"/>
  <role name="metrics" permissions="METRICS"/>
</azkaban-users>

16.启动azkaban-web-serverazkaban-exec-server

  • 启动azkaban-web-server
# 在 /home/hadoop/azkaban/azkaban-web-server 下执行:
bin/azkaban-web-start.sh
  • 启动azkaban-exec-server
# 在 /home/hadoop/azkaban/azkaban-exec-server 下执行:
bin/azkaban-exec-start.sh
  • 启动后使用jps命令,查看服务是否开启
jps

17.测试

  • 在浏览器中输入网址:
https://hadoop000:8443
  • 默认用户名:azkaban 密码:azkaban:

  • Azkaban首页:

  • 测试工程job,在本地新建一个 command.job,内容如下:
#command.job
type=command
command=echo 'hello'
  • command.job压缩成压缩包,名为command.zip

  • 点击右上角按钮:

  • 输入任务名称与描述:

  • 进入任务页面,点击右上角的upload

  • 选择之前解压好的压缩文件command.zip

  • 提交成功如下,点击Continue:

  • 任务页面,可以查看任务详情:

  • 日志信息:

以上就是Azkaban3.4的编译安装

赞(2) 打赏
未经允许不得转载:Mr. Almost的个人博客 » 编译安装Azkaban3.43

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址

谢谢老板~

支付宝扫一扫打赏

微信扫一扫打赏