Tutorial

Quickstart

Zask is easy to use:

from zask import Zask
from zask.ext.zerorpc import ZeroRPC, access_log

app = Zask(__name__)
rpc = ZeroRPC(app, middlewares=None)

@access_log
class MySrv(rpc.Server):

    def foo(self):
        return "bar"

server = MySrv()
server.bind("tcp://0.0.0.0:8081")
server.run()

Debug Mode

If config['DEBUG'] is True, then all the loggers will sent messages to stdout, otherwise, if config['DEUBG'] is False, messages will be rewrited to the files. For now, there are two loggers in the zask, zask logger and access logger.

Zask Logger

Basic Usage:

from zask import Zask

app = Zask(__name__)

app.logger.info("info")
app.logger.debug("debug")
app.logger.error("error")
try:
    raise Exception("exception")
except:
    app.logger.exception("")

Access Logger

Generally speaking, you wont use this logger directly. When you use the access_log decorator or ACCESS_LOG_MIDDLEWARE, it will working automatically.

Zask-SQLAlchemy

If you are not familiar with Flask-SQLAlchemy, the extention improves SqlAlchemy in several ways.

  1. Bind with specific framework to make it easy to use
  2. Contains all sqlalchemy and sqlalchemy.orm functions in one object
  3. Add an amazing python descriptor to the object, which make it super cool to query data
  4. Dynamic database bind depend on multiple bind configures.

As the same reason of why not just use Flask, we can’t use Flask-SQLAlchemy directly.

Differents between Flask-SQLAlchemy:

  1. Default scopefunc is gevent.getcurrent
  2. No signal session
  3. No query record
  4. No pagination and HTTP headers, e.g. get_or_404
  5. No difference between app bound and not bound

But the usage of Zask-SQLAlchemy is quite similar with Flask-SQLAlchemy. So the following of this section is just a copy from Flask-SQLAlchemy.

A Minimal Application

The SQLAlchemy provides a class called Model that is a declarative base which can be used to models:

from zask import Zask
from zask.ext.sqlalchemy import SQLAlchemy

app = Zask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True)
    email = db.Column(db.String(120), unique=True)

    def __init__(self, username, email):
        self.username = username
        self.email = email

    def __repr__(self):
        return '<User %r>' % self.username

To create the initial database, just import the db object from an interactive Python shell and run the SQLAlchemy.create_all() method to create the tables and database:

>>> from yourapplication import db
>>> db.create_all()

Boom, and there is your database. Now to create some users:

>>> from yourapplication import User
>>> admin = User('admin', 'admin@example.com')
>>> guest = User('guest', 'guest@example.com')

But they are not yet in the database, so let’s make sure they are:

>>> db.session.add(admin)
>>> db.session.add(guest)
>>> db.session.commit()

Accessing the data in database is easy as a pie:

>>> users = User.query.all()
[<User u'admin'>, <User u'guest'>]
>>> admin = User.query.filter_by(username='admin').first()
<User u'admin'>

Scope Session with ZeroRPC

Session in Zask is separated by greenlet. There is a middleware for clear session automatically:

from zask import Zask
from zask.ext.sqlalchemy import SQLAlchemy, SessionMiddleware
from zask.ext.zerorpc import ZeroRPC

app = Zask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
rpc = ZeroRPC(app)
rpc.register_middleware(SessionMiddleware(db))

Simple Relationships

SQLAlchemy connects to relational databases and what relational databases are really good at are relations. As such, we shall have an example of an application that uses two tables that have a relationship to each other:

from datetime import datetime


class Post(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(80))
    body = db.Column(db.Text)
    pub_date = db.Column(db.DateTime)

    category_id = db.Column(db.Integer, db.ForeignKey('category.id'))
    category = db.relationship('Category',
        backref=db.backref('posts', lazy='dynamic'))

    def __init__(self, title, body, category, pub_date=None):
        self.title = title
        self.body = body
        if pub_date is None:
            pub_date = datetime.utcnow()
        self.pub_date = pub_date
        self.category = category

    def __repr__(self):
        return '<Post %r>' % self.title


class Category(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(50))

    def __init__(self, name):
        self.name = name

    def __repr__(self):
        return '<Category %r>' % self.name

First let’s create some objects:

>>> py = Category('Python')
>>> p = Post('Hello Python!', 'Python is pretty cool', py)
>>> db.session.add(py)
>>> db.session.add(p)

Now because we declared posts as dynamic relationship in the backref it shows up as query:

>>> py.posts
<sqlalchemy.orm.dynamic.AppenderBaseQuery object at 0x1027d37d0>

It behaves like a regular query object so we can ask it for all posts that are associated with our test “Python” category:

>>> py.posts.all()
[<Post 'Hello Python!'>]

Multiple Databases with Binds

Zask-SQLAlchemy can easily connect to multiple databases. To achieve that it preconfigures SQLAlchemy to support multiple “binds”.

What are binds? In SQLAlchemy speak a bind is something that can execute SQL statements and is usually a connection or engine. In Flask-SQLAlchemy binds are always engines that are created for you automatically behind the scenes. Each of these engines is then associated with a short key (the bind key). This key is then used at model declaration time to assocate a model with a specific engine.

If no bind key is specified for a model the default connection is used instead (as configured by SQLALCHEMY_DATABASE_URI).

Example Configuration

The following configuration declares three database connections. The special default one as well as two others named users (for the users) and appmeta (which connects to a sqlite database for read only access to some data the application provides internally):

SQLALCHEMY_DATABASE_URI = 'postgres://localhost/main'
SQLALCHEMY_BINDS = {
    'users':        'mysqldb://localhost/users',
    'appmeta':      'sqlite:////path/to/appmeta.db'
}

Creating and Dropping Tables

The SQLAlchemy.create_all() and SQLAlchemy.drop_all() methods by default operate on all declared binds, including the default one. This behavior can be customized by providing the bind parameter. It takes either a single bind name, '__all__' to refer to all binds or a list of binds. The default bind (SQLALCHEMY_DATABASE_URI) is named None:

>>> db.create_all()
>>> db.create_all(bind=['users'])
>>> db.create_all(bind='appmeta')
>>> db.drop_all(bind=None)

Referring to Binds

If you declare a model you can specify the bind to use with the __bind_key__ attribute:

class User(db.Model):
    __bind_key__ = 'users'
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True)

Internally the bind key is stored in the table’s info dictionary as 'bind_key'. This is important to know because when you want to create a table object directly you will have to put it in there:

user_favorites = db.Table('user_favorites',
    db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
    db.Column('message_id', db.Integer, db.ForeignKey('message.id')),
    info={'bind_key': 'users'}
)

If you specified the __bind_key__ on your models you can use them exactly the way you are used to. The model connects to the specified database connection itself.

Zask-ZeroRPC

Middleware in zerorpc is designed to provide a flexible way to change the RPC behavior. Zask-ZeroRPC provides a few features by the built-in middlewares.

Configuration Based Middleware

Endpoint middleware

First is the CONFIG_ENDPOINT_MIDDLEWARE, which will resolve endpoint according to the zask application configuration. To use that you can setup a ZeroRPC like this:

app = Zask(__name__)
app.config['ZERORPC_SOME_SERVICE'] = {
    '1.0': endpoint,
}
rpc = ZeroRPC(app, middlewares=[CONFIG_ENDPOINT_MIDDLEWARE])

Then create a server and a client:

class Srv(object):
    __version__ = "1.0"
    __service_name__ = "some_service"

    def hello(self):
        return 'world'
server = rpc.Server(Srv())
# don't need bind anymore
# rpc.Server will do that for you
server.run()
client = rpc.Client('some_service', version='1.0')
client.hello()

Application will look for RPC_SOME_SERVICE config, which is combined the RPC_ prefix and upper case of some_service. You can set a default version to make the client initialization more easier:

app.config['ZERORPC_SOME_SERVICE'] = {
    '1.0': endpoint,
    'default': '1.0'
}
client = rpc.Client('some_service')
client.hello()

Custom Header Middleware

This is a default middleware.

We want to figure out where the request come from and deploy multiple version for one service. So we have to send the version and the access_key in the header, and validate the two in the server side:

app = Zask(__name__)
app.config['ZERORPC_SOME_SERVICE'] = {
    '2.0': new_endpoint,
    '1.0': old_endpoint,
    'client_keys': ['foo_client_key', 'bar_client_key'],
    'access_key': 'foo_client_key',
    'default': '2.0'
}

# as this is the default middleware
# second parameter can be omitted
rpc = ZeroRPC(app)
srv = rpc.Server(Srv())
srv.run()
client = rpc.Client('some_service')

Request header would be like this:

{
    'message_id': message_id,
    'v': 3,
    'service_name': 'some_service',
    'service_version': '2.0',
    'access_key': 'foo_client_key'
}

If access_key is not within the client_keys list of server side configuration, an exception will be raised and returned it back to the client.

But if client_keys is set to None or not setted, access_key will not be validated by the server.

Access Log Middleware

This is a default middleware.

As a RPC system, we want to save the access log for monitoring and analyzing. All the services in one physical machine will share on logfile:

'%(access_key)s - [%(asctime)s] - %(message)s'

If client don’t send access_key in the header, access_key will leave to None:

None - [2014-12-18 13:33:16,433] - "MySrv" - "foo" - OK - 1ms

Disable Middlewares

The middlewares will be applied to all the servers and clients by default. If you don’t want to use the middlewares, just set middlewares to None:

app = Zask(__name__)
rpc = ZeroRPC(app, middlewares=None)

Or set a new context to the Server/Client during the runtime:

app = Zask(__name__)
rpc = ZeroRPC(app, middlewares=[CONFIG_ENDPOINT_MIDDLEWARE])

default_context = zerorpc.Context().get_instance()
srv = rpc.Server(Srv(), context=default_context)
client = rpc.Client(context=default_context)

Default configures

Name Description
DEBUG enable/disable debug mode default: True
ERROR_LOG the path for the zask logger when DEBUG is False default: /tmp/zask.error.log
ZERORPC_ACCESS_LOG the path for the access logger when DEBUG is False default: /tmp/zask.acess.log
SQLALCHEMY_DATABASE_URI the main URI for SqlAlchemy default: sqlite://
SQLALCHEMY_BINDS multiple binds mapping. default: None
SQLALCHEMY_NATIVE_UNICODE default: None
SQLALCHEMY_ECHO enable/disable echo SqlAlchemy debug default: False
SQLALCHEMY_POOL_SIZE default: None
SQLALCHEMY_POOL_TIMEOUT default: None
SQLALCHEMY_POOL_RECYCLE default: 3600
SQLALCHEMY_MAX_OVERFLOW default: None

Best Practices

Configure loader

We have several configure files for different envs, dev and prod for example. We can load config files in a special order:

from zask import Zask

app = Zask(__name__)

# which is in the codebase
app.config.from_pyfile("settings.cfg")

# which is for development,
# ignored by codebase
app.config.from_pyfile("dev.setting.cfg", silent=True)

# which is for production,
# deployed by CM tools
app.config.from_pyfile("/etc/foo.cfg", silent=True)

# which is work with supervisord
app.config.from_envvar('CONFIG_PATH', silent=True)

# use logger after configure initialize
app.logger.debug("Config loaded")

Developer

Document

Document is powed by Sphinx. First, ensure sphinx is installed to the same environment as source code. Second, run:

$ sphinx-apidoc
$ make html

Note: update your sphinx-build to the real path in Makefile.

Testing

Similar to documentation, testing module need to be installed to the same environment. You can test one file at a time by run the script:

$ python tests/test_config.py

Or test all the cases with tox and pytest:

$ tox

Visit tox and pytest for more infomation.