00:00:00
images/kinto-logo.svg

Kinto is a minimalist JSON storage service
(store, sync, share)

Notes

Highlights

  • Why ?
  • How ?
  • About Pyramid

Notes

Why ?

Notes

Storage API

  • Universal « Web database »
  • Synchronize between devices
  • Shareable remote storage
  • Minimalist & simple

Notes

Your data, your choice

images/island-unhosted.png

Data belong to the users
(not app developers)

  • Decouple application from storage location (https://unhosted.org)
  • Self-hosting / Mutualisation
  • Location of your choice
  • Client side encryption

Notes

Synchronization

images/kinto-cloud.png
  • Live access and sharing
  • Collaborative apps
  • Offline-first
  • Progressive Web Apps

Notes

Universal means reusable!

images/overview-use-cases.png
  • Capitalize
  • Secure once
  • Deploy once
  • Scale once

Notes

New idea → Production ?

images/appdev-before.png

Notes

With Kinto:

images/appdev-after.png
  • Relax app developers!
  • Speed-up prototyping
  • Bypass backend devs/ops

Featured on Hackernews as «Self-hostable alternative to Parse and Firebase»

Notes

How ?

Notes

HTTP API

  • Hierarchy of REST resources
    (buckets > collections > records)
  • Fined-grained permission tree
images/concepts-general.png

Notes

HTTP API

  • Arbitrary JSON records
  • Polling for changes
  • Filtering + Sorting + Paginating
  • Cache and concurrency control
  • Optional JSON schema validation
  • Versioning
  • Deprecation
  • ...

Notes

Core

  • Everything pluggable from settings
    (auth, backends, ...)
  • Plain INI files + ENV vars
  • Built-in monitoring
  • HTTP API best practices

Notes

Plugins

Examples of available addons:

  • Push notifications
  • File attachments
  • History of changes
  • Digital signatures (crypto)
  • LDAP authentication
  • ...

Notes

Records storage

PostgreSQL backend (recommended):

  • SQLAlchemy (engine, pools, transactions, ...)
  • Single table with JSONB (raw SQL queries)
  • Per-request transactions
  • Minimalist DB schema migrations logic
  • Flat and easy to shard

Notes

Permissions

  • Pluggable (multi)-authentication «policies»
  • Permission backend (ACLs)
  • Intersection of «principals» (~roles)
images/concepts-permissions.png

Notes

Other pluggable backends

  • Key/value cache with «Time-To-Live»
    (Memory, Redis, ...)
  • File storage
    (filesystem, Amazon S3, ...)
  • StatsD monitoring
  • Async events/tasks
    (Redis queue)

Notes

Open source

  • 40+ contributors
  • 2800+ stargazers
  • Monthly community meetings
  • Docker image
  • 100% code coverage

Notes

Clients

  • Python (abstraction on top of requests)
from kinto_client import Client

client = Client(server="https://kinto.dev.mozaws.net/v1")
client.get_records(bucket="blog", collection="articles")
  • JavaScript (Promises + ES6)
  • Offline-first (IndexedDB)
  • Web Admin UI (React/Redux+Sagas)

Notes

kinto-admin Web UI

images/kinto-admin.png

Notes

At Mozilla

Coming soon: Web Extensions storage.sync() API

Notes

Community apps...

images/app-examples-1.png

Notes

images/app-examples-2.png

http://fourmilieres.net → Forms service!

Notes

images/app-examples-3.png

Telegram Wall idea by Xavier Orduña
PyBCN Meetup Nov 2015

Notes

About Pyramid

Notes

Key properties

  • «Pay for what you eat»
  • Very stable API
  • Good patterns
  • Flexibility
  • Extensibility

Notes

The main entry point:

from pyramid.config import Configurator

def main(**settings):
    config = Configurator(settings=settings)

    # Initialization steps using `config`.

    return config.make_wsgi_app()

With a minimalist app.wsgi file:

from myapp import main

config = configparser.ConfigParser()
config.read('config.ini')

application = main(**dict(config.items('app:main')))

Notes

Explict initialization

Imperative

  • Less magic
  • Reproductible / testable
config.add_route('hello', '/')
config.add_view(view_hello, route_name='hello')

Declarative with decorators:

@view_config(route_name='hello')
def view_hello(request):
    return {"hello": "pybcn"}

(+ explicit config.scan())

Notes

Configuration is the project «backbone»

# Map URLs to views
config.add_route()
config.add_view()

# Setup authn/authz
config.set_authentication_policy()
config.set_authorization_policy()

# Add event subscriber
config.add_subscriber()

# Add custom response renderers
config.add_renderer()
config.add_response_adapter()

# ...and more!

Notes

Extensibility

Include any package:

config.include('cornice')

Or via the settings:

pyramid.includes = webmaps_addon

Can be used to modularize any application part like views or event subscribers.

Notes

An addon is just a single Python module with a flat def includeme(config):

def includeme(config):
    # Add custom view renderer.
    config.add_renderer(name='geojson',
                        factory='webmaps.GeoJSONRenderer')

Notes

Application settings

  • Plain INI settings files
  • No sorcery at execution time
    (c.f. settings.py, conf.py)
  • OPS friendly

Notes

Python modules from settings

# config.ini
cache_backend = mypackage.cache

Easily load modules from settings files:

# main.py
settings = config.get_settings()
cache_mod_name = settings['cache_backend']

cache_module = config.maybe_dotted(cache_mod_name)
backend = cache_module.Cache(settings=settings)

Notes

Services

Declare interfaces and register components:

from pyramid.interfaces import IRoutesMapper

mapper = DummyRoutesMapper()
config.registry.registerUtility(mapper, IRoutesMapper)

Other parts of the code can query the registry:

route_mapper = request.registry.queryUtility(IRoutesMapper)
info = route_mapper(request)

Notes

Hook everything

Application initialization:

  • Powerful route/views mapping (predicates)
  • Events, callbacks, tweens, adapters, renderers, ...
  • Pyramid internals via interfaces

Notes

Events / Subscribers

class ServerFlushed(object):
    def __init__(self, request, timestamp):
        self.request = request
        self.timestamp = timestamp

Trigger event from view:

from .events import ServerFlushed

def view_flush_post(request):
    request.registry.storage.flush()

    event = ServerFlushed(request, timestamp=datetime.now())
    request.registry.notify(event)

    return {"status": "ok"}

Notes

Subscribe to event during initialization:

def on_server_flush(event):
    request = event.request
    # Add header to response
    request.response.headers['Alert'] = 'Flush'

config.add_subscriber(on_server_flush, ServerFlushed)
  • Alter responses
  • Raise HTTP exceptions (eg. quotas, etc.)

Executed synchronously → use job queue for long tasks

Notes

Custom abstractions

Example of domain specific initialization method:

def add_api_capability(config, identifier, description=""):
    capability = dict(description=description)
    # The application registry is a singleton
    config.registry.api_capabilities[identifier] = capability

config.add_directive('add_api_capability', add_api_capability)

New initialization directive becomes available:

config.add_api_capability('history', description="History plugin")

Notes

This view exposes what plugins have registered via our custom method:

@view_config(route_name='hello')
def get_hello(request):
    data = {
        'capabilities': request.registry.api_capabilities
    }
    return data

Craft your own special-purpose, domain-specific Web system → «framework framework»

Notes

Testing

from myapp import main

class PluginSetupTest(unittest.TestCase):
    settings = {
        'pyramid.includes': 'extra_plugin'
    }

    def __init__(self, *args, **kwargs):
        super(WebTest, self).__init__(*args, **kwargs)
        wsgi_app = testapp(self.settings)
        self.app = webtest.TestApp(wsgi_app)
        self.headers = {"Content-Type": "application/json"}

    def test_capability_is_shown_in_hello_view(self):
        resp = self.app.get("/hello", headers=self.headers)
        assert "extra_plugin" in resp.json["capabilities"]

Notes

Compose vs. inherit

class MyAuthz(Authorization):
    def permits(self):
        permits = super(MyAuthz, self).permits()
        return permits and custom_check()

With inheritance, substitution occurs before instantiation.

With composition, we can do:

class Authorization:
    def permits(self):
        return self.context.is_allowed()

authz.context = MyContext()

Notes

Prefer composition because:

  • Readability
  • Flexibility
  • Single responsability principle
  • Composition of uncoupled packages
  • Avoid multiple inheritance (eg. mixins)

Notes

Downsides

  • Pyramid is not the «latest cool stuff»
  • Projects not always active (but just works)
  • Documentation lacks «real-life examples»
    (e.g. ACL)
  • Easy to couple everything to request
  • Built-in authentication policies are not intuitive

Notes

Gràcies!

Notes

Notes