Learn-RabbitMQ.pdf - Google Drive

1 downloads 4 Views 763KB Size Report
Container persistence via disk automount on /var/lib/docker. } $ docker -v. } User/Passwd: docker/tcuser (192.168.59.103
Learn RabbitMQ with Python in 90 minutes Larry cai

Agenda }  }  }  }  }  }  } 

https://www.rabbitmq.com/ Introduction Exercise 1: Hello World Exercise 2: Work queues - Direct Exercise 3: Publish/Subscribe - Fanout Exercise 4: Routing - Direct Exercise 5: Topics Reference

Mostly are based https://www.rabbitmq.com/getstarted.html 2

7/22/15

Environment Preparation (docker) } 

Boot2docker Installer (127M) } 

Contains latest docker already, fast

} 

Container persistence via disk automount on /var/lib/docker

} 

$ docker -v User/Passwd: docker/tcuser (192.168.59.103)

} 

} 

Download python/rabbitmq:2 docker images }  } 

} 

http://boot2docker.io/

$ docker pull python:2 $ docker pull rabbitmq:3-management

(Windows/Mac) Clone the code from github to your user directory (~/git) https://github.com/larrycai/codingwithme-rabbitmq

} 

Notepad++ & MobaXterm are recommended 3

7/22/15

Introduction RabbitMQ is a lightweight, reliable, scalable and portable message broker. (Apache ActiveMQ/Apollo, ZeroMQ) }  It supports AMQP (Advanced Message Queuing Protocol) }  Written in Erlang/OTP }  Messaging: Pattern (asynchronous) } 

}  }  } 

4

Producer: create message Consumer: Handle a message Queue: Stack the message

7/22/15

Exercise 1: Hello World } 

Run first app inside docker environment $ cd /c/Users//git/codingwithme-rabbitmq $ docker run -d -e RABBITMQ_NODENAME=rabbit --name rabbit -p 15672:15672 rabbitmq:3-management $ docker run -v $PWD:/code -w /code --name worker --link=rabbit:rabbit -it python:2 bash # pip install pika

Python:2 (worker)

# cd exer1 && ./send.py

} 

} 

Browse: http://192.168.59.103:15672 } See overview and Check Exchange/Queue } Get message

Run in another shell inside docker

15672

(rabbit)

Docker Host

$ docker exec –it worker bash # cd exer1 && ./receive.py

} Check } 

queue again in GUI Update the “Hello world” to “Hello Larry”

5

RabbitMQ Server

7/22/15

RabbitMQ - Concept } 

Publisher/Consumer is app (running in python container in exercise 1) } 

} 

Publish & consume the messages

In RabbitMQ, An exchange receives messages from producers and pushes them to queues.

6

7/22/15

Rabbit Libraries - Python Pika module }  } 

Pika is RabbitMQ server python module Procedure for Publish 1. 

Establish connection

2. 

Define Queue (“Hello”) Publish (routing_key is queue name)

3. 

} 

Consume }  } 

7

1,2 same for publish 3: Call back

7/22/15

Queue in depth } 

Round-robin dispatching } 

} 

Message acknowledgment } 

} 

Scale to distribute to different consumers An ack(nowledgement) is sent back from the consumer to tell RabbitMQ that a particular message had been received

Message durability & Fair dispatch (self-learning) 8

7/22/15

Exercise 2: Work queues } 

How to scale the twitter message image processing twitter.py – publish twitter message

} 

image.py – process the image if twitter has

} 

Env

} 

# cd ../exer2 #./twitter.py “I see the nice picture 1.png” #./image.py

Did we receive the message in queue ? Why ? }  Run more image.py to see how it is distributed (another shell, docker exec -it .. ) }  Kill some image.py } 

9

7/22/15

Exchange types Exchange type is the rule on how to deliver message to the queues: direct, fanout, topic and headers }  Direct: if the routing key matches, then the message is delivered to the corresponding queue. }  Fanout: multicast the received message to the bound queues. } 

}  } 

10

Publish/Subscribe Patten Broadcast

7/22/15

Exercise 3: Fanout - Publish/Subscribe } 

} 

How to broadcast the message for different consumers }  twitter.py – publish twitter message }  image.py – manipulate the image if twitter has Env # cd ../exer3 #./twitter.py “I see the nice picture 1.png” #./image.py

}  } 

Did we receive first twitter & Check in the browser for the queue Implement extra tag.py check whether twitter has tag }  }  } 

11

Define queue `tag` and bind to `twitter` exchange Search for “#xxx#” keywords and print it out ./twitter.py “#codingwithme# is good, send picture 1.png” 7/22/15

Binding A binding is a relationship between an exchange and a queue.  }  One to one mapping } 

} 

Multiple mapping with

12

Routing_key

7/22/15

Exercise 4: Direct - Routing }  } 

How to handle different log system Env # cd ../exer4 # ./emit_log_direct.py # ./receive_logs_direct.py

} 

Exercise:

} 

Check GUI for queue name and bindings

} 

More receiver # ./receive_logs_direct.py warning error > logs_from_rabbit.log # ./receive_logs_direct.py info warning error

} 

Define fixed queue name

13

7/22/15

Topics }  } 

User pattern as routing key (“a.b.c.d”) Queues can use wildcard characters in binding }  } 

} 

* matches a single word, # zero or more words. “rabbit.*” matches for rabbit.info/rabbit.error

Topic exchange is powerful and can behave like other exchanges. }  } 

14

When a queue is bound with "#" (hash) binding key - it will receive all the messages, regardless of the routing key - like in fanout exchange. When special characters "*" (star) and "#" (hash) aren't used in bindings, the topic exchange will behave just like a direct one.

7/22/15

Exercise 5: Topic }  } 

handle different log system more Env # cd ../exer5 # ./emit_log_direct.py # ./receive_logs_direct.py “#”

} 

Exercise }  }  } 

Check the queue name in GUI Create for rabbit.*/apache.*/*.info Publish message for rabbit.info/error & apache.info/ error

15

7/22/15

Summary } 

RabbitMQ is the message broker support AMQP } 

} 

Handle async messages

AMQP Model }  }  }  } 

16

Different Exchange types Messages Queues Bindings Rule for Binding ..

7/22/15

Reference }  }  } 

http://www.rabbitmq.com/ Exercises are based on http://www.rabbitmq.com/getstarted.html Book: } 

} 

RabbitMQ in Action (2012): http://www.manning.com/videla/

Code: } 

17

https://github.com/larrycai/codingwithme-rabbitmq

7/22/15

ChangeLog } 

2015/05/11: first release

18

7/22/15