1 Jar包准备

1) 将MySql驱动包放入Flume的lib目录下

[atguigu@hadoop102 flume]$ cp \

/opt/sorfware/mysql-libs/mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar \

/opt/module/flume/lib/

2) 打包项目并将Jar包放入Flume的lib目录下

2 配置文件准备

1）创建配置文件并打开

[atguigu@hadoop102 job]$ touch mysql.conf

[atguigu@hadoop102 job]$ vim mysql.conf

2）添加如下内容

# Name the components on this agent

a1.sources = r1

a1.sinks = k1

a1.channels = c1

 

# Describe/configure the source

a1.sources.r1.type = com.atguigu.source.SQLSource 

a1.sources.r1.connection.url = jdbc:mysql://192.168.9.102:3306/mysqlsource

a1.sources.r1.connection.user = root 

a1.sources.r1.connection.password = 000000 

a1.sources.r1.table = student 

a1.sources.r1.columns.to.select = * 

#a1.sources.r1.incremental.column.name = id 

#a1.sources.r1.incremental.value = 0

a1.sources.r1.run.query.delay=5000

 

# Describe the sink

a1.sinks.k1.type = logger

 

# Describe the channel

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

 

# Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

3 MySql表准备

1) 创建MySqlSource数据库

CREATE DATABASE mysqlsource；

2) 在MySqlSource数据库下创建数据表Student和元数据表Flume_meta

CREATE TABLE `student` (

`id` int(11) NOT NULL AUTO_INCREMENT,

`name` varchar(255) NOT NULL,

PRIMARY KEY (`id`)

);

CREATE TABLE `flume_meta` (

`source_tab` varchar(255) NOT NULL,

`currentIndex` varchar(255) NOT NULL,

PRIMARY KEY (`source_tab`)

);

• 向数据表中添加数据

1 zhangsan

2 lisi

3 wangwu

4 zhaoliu

4  测试并查看结果

• 任务执行

[atguigu@hadoop102 flume]$ bin/flume-ng agent --conf conf/ --name a1 \

--conf-file job/mysql.conf -Dflume.root.logger=INFO,console

• 结果展示，如图6-2所示：

想要了解跟多关于大数据培训机构课程内容欢迎关注尚硅谷大数据培训机构，尚硅谷除了这些技术文章外还有免费的高质量大数据培训机构课程视频供广大学员下载学习