Versions

This document provides migration instructions for each major version, as well as the complete changelog for versions dating back to v0.9.0 from December 2015. The migration guides provide detailed examples and tips for migrating from the previous major version (excluding the 1.0.0 guide, which only covers migration from 0.9.0 and newer).

Migrating to 3.0.0

The 3.0.0 release includes two api changes from 2.4.0 that you may need to update your code to handle.

• The atomic= kwarg to Engine.save and Engine.delete was deprecated in 2.4.0 and is removed in 3.0.0.

• The return type of Type._dump must now be a bloop.action.Action instance, even when the value is None. This does not impact custom types that only implement dynamo_load and dynamo_dump.

atomic keyword

The atomic keyword to Engine.save and Engine.delete has been removed in favor of a user pattern. This offers a reasonable performance improvement for users that never used the atomic keyword, and addresses ambiguity related to per-row atomic vs transactional atomic operations. For context on the deprecation, see Issue #138. For the equivalent user pattern, see Snapshot Condition. To migrate your existing code, you can use the following:

# pre-3.0 code to migrate:
engine.load(some_object)
some_object.some_attr = "new value"
engine.save(some_object, atomic=True)

# post-3.0 code:

# https://bloop.readthedocs.io/en/latest/user/patterns.html#snapshot-condition
from your_patterns import snapshot

engine.load(some_object)
condition = snapshot(some_object)
some_object.some_attr = "new value"
engine.save(some_object, condition=condition)

Type._dump

Bloop now allows users to specify how a value should be applied in an UpdateExpression by wrapping a value in a bloop.actions.Action object. This is done transparently for raw values, which are interpreted as either bloop.actions.set or bloop.actions.remove. With 2.4 and to support Issue #136 you can also specify an add or delete action:

my_user.aliases = bloop.actions.add("new_alias")
my_website.views = bloop.actions.add(1)

To maintain flexibility the bloop Type class has the final say as to which action a value should use. This allows eg. the List type to take a literal [] and change the action from actions.set to actions.remove(None) to indicate that the value should be cleared. This also means your custom type could see an actions.delete and modify the value to instead be expressible in an actions.set.

If your custom types today only override dynamo_dump or dynamo_load then you don't need to do anything for this migration. However if you currently override _dump then you should update your function to (1) handle input that may be an action or not, and (2) always return an action instance. In general, you should not modify an input action and instead should return a new instance (possibly with the same action_type).

Here's the migration of the base Type._dump:

# pre-3.0 code to migrate:
def _dump(self, value, **kwargs):
    value = self.dynamo_dump(value, **kwargs)
    if value is None:
        return None
    return {self.backing_type: value}


# post-3.0 code:
from bloop import actions

def _dump(self, value, **kwargs):
    wrapped = actions.wrap(value)  # [1]
    value = self.dynamo_dump(wrapped.value, **kwargs)
    if value is None:
        return actions.wrap(None)  # [2]
    else:
        value = {self.backing_type: value}
        return wrapped.type.new_action(value)  # [3]

# [1] always wrap the input value to ensure you're working with an Action instance
# [2] returns actions.remove(None) which will remove the value like None previously did
# [3] new_action uses the **same action type** as the input.
#         If you want to always return a SET action instead use: return actions.set(value)

Migrating to 2.0.0

The 2.0.0 release includes a number of api changes and new features.

• The largest functional change is the ability to compose models through subclassing; this is referred to as Abstract Inheritance and Mixins throughout the User Guide.

• Python 3.6.0 is the minimum required version.

• Meta.init now defaults to cls.__new__(cls) instead of cls.__init__(); when model instances are created as part of engine.query, engine.stream etc. these will not call your model's __init__ method. The default BaseModel.__init__ is not meant for use outside of local instantiation.

• The Column and Index kwarg name was renamed to dynamo_name to accurately reflect how the value was used: Column(SomeType, name="foo") becomes Column(SomeType, dynamo_name="foo"). Additionally, the column and index attribute model_name was renamed to name; dynamo_name is unchanged and reflects the kwarg value, if provided.

Engine

A new Engine kwarg table_name_template can be used to modify the table name used per-engine, as documented in the new Engine Configuration section of the User Guide. Previously, you may have used the before_create_table signal as follows:

# Nonce table names to avoid testing collisions
@before_create_table.connect
def apply_table_nonce(_, model, **__):
    nonce = datetime.now().isoformat()
    model.Meta.table_name += "-test-{}".format(nonce)

This will modify the actual model's Meta.table_name, whereas the new kwarg can be used to only modify the bound table name for a single engine. The following can be expressed for a single Engine as follows:

def apply_nonce(model):
    nonce = datetime.now().isoformat()
    return f"{model.Meta.table_name}-test-{nonce}"

engine = Engine(table_name_template=apply_nonce)

Inheritance

You can now use abstract base models to more easily compose common models. For example, you may use the same id structure for classes. Previously, this would look like the following:

class User(BaseModel):
    id = Column(String, hash_key=True)
    version = Column(Integer, range_key=True)
    data = Column(Binary)

class Profile(BaseModel):
    id = Column(String, hash_key=True)
    version = Column(Integer, range_key=True)
    summary = Column(String)

Now, you can define an abstract base and re-use the id and version columns in both:

class MyBase(BaseModel):
    class Meta:
        abstract = True
    id = Column(String, hash_key=True)
    version = Column(Integer, range_key=True)

class User(MyBase):
    data = Column(Binary)

class Profile(MyBase):
    summary = Column(String)

You can use multiple inheritance to compose models from multiple mixins; base classes do not need to subclass BaseModel. Here's the same two models as above, but the hash and range keys are defined across two mixins:

class StringHash:
    id = Column(String, hash_key=True)

class IntegerRange:
    version = Column(Integer, range_key=True)


class User(StringHash, IntegerRange, BaseModel):
    data = Column(Binary)

class Profile(StringHash, IntegerRange, BaseModel):
    summary = Column(String)

Mixins may also contain GlobalSecondaryIndex and LocalSecondaryIndex, even if their hash/range keys aren't defined in that mixin:

class ByEmail:
    by_email = GlobalSecondaryIndex(projection="keys", hash_key="email")


class User(StringHash, IntegerRange, ByEmail, BaseModel):
    email = Column(String)

Meta.init

With the addition of column defaults (see below) Bloop needed to differentiate local mode instantiation from remote model instantiation. Local model instantiation still uses __init__, as in:

user = User(email="me@gmail.com", verified=False)

Unlike Engine.load which takes existing model instances, all of Engine.query, Engine.scan, Engine.stream will create their own instances. These methods use the model's Meta.init to create new instances. Previously this defaulted to __init__. However, with the default __init__ method applying defaults in 2.0.0 this is no longer acceptable for remote instantiation. Instead, cls.__new__(cls) is used by default to create instances during query/scan/stream.

This is an important distinction that Bloop should have made early on, but was forced due to defaults. For example, imagine querying an index that doesn't project a column with a default. If the base __init__ was still used, the Column's default would be used for the non-projected column even if there was already a value in DynamoDB. Here's one model that would have the problem:

class User(BaseModel):
    id = Column(UUID, hash_key=True)
    created = Column(DateTime, default=datetime.datetime.now)
    email = Column(String)
    by_email = GlobalSecondaryIndex(projection="keys", hash_key=email)

user = User(id=uuid.uuid4(), email="me@gmail.com")
engine.save(user)
print(user.created)  # Some datetime T1


query = engine.Query(User.by_email, hash_key=User.email=="me@gmail.com")
partial_user = query.first()

partial_user.created  # This column isn't part of the index's projection!

If User.Meta.init was still User.__init__ then partial_user.created would invoke the default function for User.created and give us the current datetime. Instead, Bloop 2.0.0 will call User.__new__(User) and we'll get an AttributeError because partial_user doesn't have a created value.

Column Defaults

Many columns have the same initialization value, even across models. For example, all but one of the following columns will be set to the same value or using the same logic:

class User(BaseModel):
    email = Column(String, hash_key=True)
    id = Column(UUID)
    verified = Column(Boolean)
    created = Column(DateTime)
    followers = Column(Integer)

Previously, you might apply defaults by creating a simple function:

def new_user(email) -> User:
    return User(
        email=email,
        id=uuid.uuid4(),
        verified=False,
        created=datetime.datetime.now(),
        followers=0)

You'll still need a function for related initialization (eg. across fields or model instances) but for simple defaults, you can now specify them with the Column:

class User(BaseModel):
    email = Column(String, hash_key=True)
    id = Column(UUID, default=uuid.uuid4)
    verified = Column(Boolean, default=False)
    created = Column(DateTime, default=datetime.datetime.now)
    followers = Column(Integer, default=0)


def new_user(email) -> User:
    return User(email=email)

Defaults are only applied when creating new local instances inside the default BaseModel.__init__ - they are not evaluated when loading objects with Engine.load, Engine.query, Engine.stream etc. If you define a custom __init__ without calling super().__init__(...) they will not be applied.

In a related change, see above for the BaseModel.Meta.init change. By default Bloop uses cls.__new__(cls) to create new instances of your models during Engine.scan and Engine.query instead of the previous default to __init__. This is intentional, to avoid applying unnecessary defaults to partially-loaded objects.

TTL

DynamoDB introduced the ability to specify a TTL column, which indicates a date (in seconds since the epoch) after which the row may be automatically (eventually) cleaned up. This column must be a Number, and Bloop exposes the Timestamp type which is used as a datetime.datetime. Like the DynamoDBStreams feature, the TTL is configured on a model's Meta attribute:

class TemporaryPaste(BaseModel):
    class Meta:
        ttl = {
            "column": "delete_after"
        }
    id = Column(String, hash_key=True)
    s3_location = Column(String, dynamo_name="s3")
    delete_after = Column(Timestamp)

Remember that it can take up to 24 hours for the row to be deleted; you should guard your reads using the current time against the cleanup time, or a filter with your queries:

# made up index
query = engine.Query(
    TemporaryPaste.by_email,
    key=TemporaryPaste.email=="me@gmail.com",
    filter=TemporaryPaste.delete_after <= datetime.datetime.now())
print(query.first())

Bloop still refuses to update existing tables, so TTL will only be enabled on tables if they are created by Bloop during Engine.bind. Otherwise, the declaration exists exclusively to verify configuration.

Types

A new type Timestamp was added for use with the new TTL feature (see above). This is a datetime.datetime in Python just like the DateTime type, but is stored as an integer (whole seconds since epoch) instead of an ISO 8601 string. As with DateTime, drop-in replacements are available for arrow, delorean, and pendulum.

Exceptions

• InvalidIndex was replaced by the existing InvalidModel

• InvalidSearchMode, InvalidKeyCondition, InvalidFilterCondition, and InvalidProjection were replaced by InvalidSearch

• UnboundModel was removed without replacement; Engine.bind was refactored so that it would never be raised.

• InvalidComparisonOperator was removed without replacement; it was never raised.

Migrating to 1.0.0

The 1.0.0 release includes a number of api changes, although functionally not much has changed since 0.9.6. The biggest changes are to Query and Scan syntax, which has changed from a builder pattern to a single call. The remaining changes are mostly resolved through a different import or parameter/attribute name.

Session, Client

In 1.0.0 the Engine wraps two clients: one for DynamoDB, and one for DynamoDBStreams. Bloop will create default clients for any missing parameters using boto3.client:

import boto3
from bloop import Engine

ddb = boto3.client("dynamodb")
streams = boto3.client("dynamodbstreams")
engine = Engine(dynamodb=ddb, dynamodbstreams=streams)

Before 0.9.11

Prior to 0.9.11, you could customize the session that an Engine used to talk to DynamoDB by creating an instance of a boto3.session.Session and passing it to the Engine during instantiation. This allowed you to use a different profile name:

from boto3 import Session
from bloop import Engine

session = Session(profile_name="my-profile")
engine = Engine(session=session)

Now, you will need to create client instances from that session:

from boto3 import session
from bloop import Engine

session = Session(profile_name="my-profile")
engine = Engine(
    dynamodb=session.client("dynamodb"),
    dynamodbstreams=session.client("dynamodbstreams")
)

After 0.9.11

In 0.9.11, the Engine changed to take a bloop.Client which wrapped a boto3 client. This allowed you to connect to a different endpoint, such as a DynamoDBLocal instance:

import boto3
from bloop import Client, Engine

boto_client = boto3.client("dynamodb", endpoint_url="http://localhost:8000")
bloop_client = Client(boto_client=boto_client)
engine = Engine(client=bloop_client)

The intermediate bloop Client is no longer necessary, but a dynamodbstreams client can be provided:

import boto3
from bloop import Client, Engine

ddb = boto3.client("dynamodb", endpoint_url="http://localhost:8000")
streams = boto3.client("dynamodbstreams", endpoint_url="http://localhost:8000")
engine = Engine(dynamodb=ddb, dynamodbstreams=streams)

Engine

Config

Prior to 1.0.0, Engine took a number of configuration options. These have all been removed, and baked into existing structures, or are only specified at the operation level. Engine no longer takes **config kwargs.

• atomic controlled the default value for delete and save operations. If your engine had a default atomic of True, you must now explicitly specify that with each delete and save. The same is true for consistent, which controlled the default for load, query, and scan.

• prefetch controlled the default number of items that Bloop would fetch for a query or scan. Bloop now uses the built-in pagination controls, and will fetch the next page when the currently buffered page has been iterated. There is no way to control the number of items loaded into the buffer at once.

• strict controlled the default setting for query and scan against an LSI. This is now part of the declaration of an LSI: by_create = LocalSecondaryIndex(projection="all", range_key="created", strict=False). By default an LSI is strict, which matches the default configuration option. This change means an LSI must be accessed by every caller the same way. You can't have one caller use strict=True while another uses strict=False.

EngineView and context

Because there are no more engine.config values, there is no point to using engines as context managers. Previously, you could use an EngineView to change one config option of an engine for a local command, without changing the underlying engine's configuration:

with engine.context(atomic=True) as atomic:
    atomic.save(...)
    # a bunch of operations that perform atomic saves

Engine.context and the EngineView class have been removed since there is no longer an Engine.config.

Engine.save, Engine.delete

These functions take *objs instead of objs, which makes passing a small number of items more comfortable.

user = User(...)
tweet = Tweet(...)

# Old: explicit list required
engine.save([user, tweet])

# 1.0.0: *varargs
engine.save(user, tweet)

# 1.0.0: save a list
some_users = get_modified()
engine.save(*some_users)

Query, Scan

Queries and Scans are now created in a single call, instead of using an ambiguous builder pattern. This will simplify most calls, but will be disruptive if you rely on partially building queries in different parts of your code.

Creating Queries

The most common issue with the builder pattern was creating multi-condition filters. Each call would replace the existing filter, not append to it. For example:

# This only checks the date, NOT the count
q = engine.query(User).key(User.id == 0)
q = q.filter(User.friends >= 3)
q = q.filter(User.created >= arrow.now().replace(years=-1))

# 1.0.0 only has one filter option
q = engine.query(
    User, key=User.id == 0,
    filter=(
        (User.friends >= 3) &
        (User.created >= ...)
    )
)

The other query controls have been baked in, including projection, consistent, and forward. Previously, you changed the forward option through the properties ascending and descending. Use forward=False to sort descending.

Here is a query with all options before and after. The structure is largely the same, with a lower symbolic overhead:

# Pre 1.0.0
q = (
    engine.query(User)
        .key(User.id == 0)
        .projection("all")
        .descending
        .consistent
        .filter(User.name.begins_with("a"))
)

# 1.0.0
q = engine.query(
    User,
    key=User.id == 0,
    projection="all",
    forward=False,
    consistent=True,
    filter=User.name.begins_with("a")
)

The same changes apply to Engine.scan, although Scans can't be performed in descending order.

Parallel Scans

1.0.0 allows you to create a parallel scan by specifying the segment that this scan covers. This is just a tuple of (Segment, TotalSegments). For example, to scan Users in three pieces:

scans = [
    engine.scan(User, parallel=(0, 3)),
    engine.scan(User, parallel=(1, 3)),
    engine.scan(User, parallel=(2, 3))
]

for worker, scan in zip(workers, scans):
    worker.process(scan)

Iteration and Properties

The all method and prefetch and limit options have been removed. Each call to Engine.query() or Engine.scan() will create a new iterator that tracks its progress and can be reset. To create different iterators over the same parameters, you must call Engine.query() multiple times.

# All the same iterator
>>> scan = engine.scan(User, filter=...)
>>> it_one = iter(scan)
>>> it_two = iter(scan)
>>> it_one is it_two is scan
True

Query and Scan no longer buffer their results, and you will need to reset the query to execute it again.

>>> scan = engine.scan(User)
>>> for result in scan:
...     pass
...
>>> scan.exhausted
True
>>> scan.reset()
>>> for result in scan:
...     print(result.id)
...
0
1
2

• The complete property has been renamed to exhausted to match the new Stream interface.

• The results property has been removed.

• count, scanned, one(), and first() are unchanged.

Models

Base Model and abstract

Model declaration is largely unchanged, except for the model hierarchy. Early versions tied one base model to one engine; later versions required a function to create each new base. In 1.0.0, every model inherits from a single abstract model, BaseModel:

from bloop import BaseModel, Column, Integer


class User(BaseModel):
    id = Column(Integer, hash_key=True)
    ...

Additionally, any model can be an abstract base for a number of other models (to simplify binding subsets of all models) by setting the Meta attribute abstract to True:

from bloop import BaseModel

class AbstractUser(BaseModel):
    class Meta:
        abstract = True

    @property
    def is_email_verified(self):
        return bool(getattr(self, "verified", False))

Before 0.9.6

Models were tied to a single Engine, and so the base class for any model had to come from that Engine:

from bloop import Engine

primary = Engine()
secondary = Engine()

class User(primary.model):
    ...

# secondary can't save or load instances of User!

Now that models are decoupled from engines, any engine can bind and load any model:

from bloop import BaseModel, Engine

primary = Engine()
secondary = Engine()

class User(BaseModel):
    ...

primary.bind(User)
secondary.bind(User)

After 0.9.6

After models were decoupled from Engines, Bloop still used some magic to create base models that didn't have hash keys but also didn't fail various model creation validation. This meant you had to get a base model from new_base():

from bloop import Engine, new_base

primary = Engine()
secondary = Engine()

Base = new_base()

class User(Base):
    ...

primary.bind(User)
secondary.bind(User)

Now, the base model is imported directly. You can simplify the transition using an alias import. To adapt the above code, we would alias BaseModel to Base:

from bloop import Engine
from bloop import BaseModel as Base

primary = Engine()
secondary = Engine()

class User(Base):
    ...

primary.bind(User)
secondary.bind(User)

Binding

Engine.bind has undergone a few stylistic tweaks, and started offering recursive binding. The parameter base is no longer keyword-only.

To bind all concrete (Meta.abstract=False) models from a single base, pass the base model:

from bloop import BaseModel, Engine

class AbstractUser(BaseModel):
    class Meta:
        abstract = True

class AbstractDataBlob(BaseModel):
    class Meta:
        abstract = True

class User(AbstractUser):
    ...

class S3Blob(AbstractDataBlob):
    ...

engine = Engine()
engine.bind(AbstractUser)

This will bind User but not S3Blob.

Indexes

Projection is Required

In 1.0.0, projection is required for both GlobalSecondaryIndex and LocalSecondaryIndex. This is because Bloop now supports binding multiple models to the same table, and the "all" projection is not really DynamoDB's all, but instead an INCLUDE with all columns that the model defines.

Previously:

from bloop import new_base, Column, Integer, GlobalSecondaryIndex

class MyModel(new_base()):
    id = Column(Integer, hash_key=True)
    data = Column(Integer)

    # implicit "keys"
    by_data = GlobalSecondaryIndex(hash_key="data")

Now, this must explicitly state that the projection is "keys":

from bloop import BaseModel, Column, Integer, GlobalSecondaryIndex

class MyModel(BaseModel):
    id = Column(Integer, hash_key=True)
    data = Column(Integer)

    by_data = GlobalSecondaryIndex(
        projection="keys", hash_key="data")

Hash and Range Key

1.0.0 also lets you use the Column object (and not just its model name) as the parameter to hash_key and range_key:

class MyModel(BaseModel):
    id = Column(Integer, hash_key=True)
    data = Column(Integer)

    by_data = GlobalSecondaryIndex(
        projection="keys", hash_key=data)

__set__ and __del__

Finally, Bloop disallows setting and deleting attributes on objects with the same name as an index. Previously, it would simply set that value on the object and silently ignore it when loading or saving. It wasn't clear that the value wasn't applied to the Index's hash or range key.

>>> class MyModel(BaseModel):
...     id = Column(Integer, hash_key=True)
...     data = Column(Integer)
...     by_data = GlobalSecondaryIndex(
...         projection="keys", hash_key=data)
...
>>> obj = MyModel()
>>> obj.by_data = "foo"
Traceback (most recent call last):
  ...
AttributeError: MyModel.by_data is a GlobalSecondaryIndex

Types

DateTime

Previously, DateTime was backed by arrow. Instead of forcing a particular library on users -- and there are a number of high-quality choices -- Bloop's built-in datetime type is now backed by the standard library's datetime.datetime. This type only loads and dumps values in UTC, and uses a fixed ISO8601 format string which always uses +00:00 for the timezone. DateTime will forcefully convert the timezone when saving to DynamoDB with datetime.datetime.astimezone() which raises on naive datetime objects. For this reason, you must specify a timezone when using this type.

Most users are expected to have a preferred datetime library, and so Bloop now includes implementations of DateTime in a new extensions module bloop.ext for the three most popular datetime libraries: arrow, delorean, and pendulum. These expose the previous interface, which allows you to specify a local timezone to apply when loading values from DynamoDB. It still defaults to UTC.

To swap out an existing DateTime class and continue using arrow objects:

# from bloop import DateTime
from bloop.ext.arrow import DateTime

To use delorean instead:

# from bloop import DateTime
from bloop.ext.delorean import DateTime

Future extensions will also be grouped by external package, and are not limited to types. For example, an alternate Engine implementation could be provided in bloop.ext.sqlalchemy that can bind SQLAlchemy's ORM models and transparently maps Bloop types to SQLALchemy types.

Float

Float has been renamed to Number and now takes an optional decimal.Context to use when translating numbers to DynamoDB's wire format. The same context used in previous versions (which comes from the specifications in DynamoDB's User Guide) is used as the default; existing code only needs to use the new name or alias it on import:

# from bloop import Float
from bloop import Number as Float

A new pattern has been added that provides a less restrictive type which always loads and dumps float instead of decimal.Decimal. This comes at the expense of exactness, since Float's decimal context does not trap Rounding or Inexact signals. This is a common request for boto3; keep its limitations in mind when storing and loading values. It's probably fine for a cached version of a product rating, but you're playing with fire storing account balances with it.

String

A minor change, String no longer calls str(value) when dumping to DynamoDB. This was obscuring cases where the wrong value was provided, but the type silently coerced a string using that object's __str__. Now, you will need to manually call str on objects, or boto3 will complain of an incorrect type.

>>> from bloop import BaseModel, Column, Engine, String

>>> class MyModel(BaseModel):
...     id = Column(String, hash_key=True)
...
>>> engine = Engine()
>>> engine.bind(MyModel)

>>> not_a_str = object()
>>> obj = MyModel(id=not_a_str)

# previously, this would store "<object object at 0x7f92a5a2f680>"
# since that is str(not_a_str).
>>> engine.save(obj)

# now, this raises (newlines for readability)
Traceback (most recent call last):
  ..
ParamValidationError: Parameter validation failed:
Invalid type for
    parameter Key.id.S,
    value: <object object at 0x7f92a5a2f680>,
    type: <class 'object'>,
    valid types: <class 'str'>

Exceptions

NotModified was raised by Engine.load when some objects were not found. This has been renamed to MissingObjects and is otherwise unchanged.

Exceptions for unknown or abstract models have changed slightly. When an Engine fails to load or dump a model, it will raise UnboundModel. When a value fails to load or dump but isn't a subclass of BaseModel, the engine raises UnknownType. When you attempt to perform a mutating operation (load, save, ...) on an abstract model, the engine raises InvalidModel.

Changelog

This changelog structure is based on Keep a Changelog v0.3.0. Bloop follows Semantic Versioning 2.0.0 and a draft appendix for its Public API.

Unreleased

(no unreleased changes)

3.1.0 - 2021-11-11

Fixed an issue where copying an Index would lose projection information when the projection mode was "include". This fix should have no effect for most users. You would only run into this issue if you were manually calling bind_index with copy=True on a projection mode "include" or you subclass a model that has an index with that projection mode. This does not require a major version change since there is no reasonable workaround that would be broken by making this fix. For example, a user might decide to monkeypatch Index.__copy__, bind_index or refresh_index to preserve the projection information. Those workarounds will not be broken by this change. For an example of the issue, see Issue #147.

[Changed]

• Index.projection is now a set instead of a list`.  Since ``Column implements __hash__ this won't affect any existing calls that pass in lists. To remain consistent, this change is reflected in Engine.search, Search.__init__, Index.__init__, and any docs or examples that refer to passing lists/sets of Columns.

[Fixed]

• Index.__copy__ preserves Index.projection["included"] when projection mode is "include".

3.0.0 - 2019-10-11

Remove deprecated keyword atomic= from Engine.save and Engine.delete, and Type._dump must return a bloop.actions.Action instance. See the Migration Guide for context on these changes, and sample code to easily migrate your existing custom Types.

[Added]

• (internal) util.default_context can be used to create a new load/dump context and respects existing dict objects and keys (even if empty).

[Changed]

• Type._dump must return a bloop.actions.Action now. Most users won't need to change any code since custom types usually override dynamo_dump. If you have implemented your own _dump function, you can probably just use actions.wrap and actions.unwrap to migrate:

  def _dump(self, value, *, context, **kwargs):
      value = actions.unwrap(value)
      # the rest of your function here
      return actions.wrap(value)
  

[Removed]

• The deprecated atomic= keyword has been removed from Engine.save and Engine.delete.

• The exception bloop.exceptions.UnknownType is no longer raised and has been removed.

• (internal) BaseModel._load and BaseModel._dump have been removed. These were not documented or used anywhere in the code base, and unpack_from_dynamodb should be used where _load was anyway.

• (internal) Engine._load and Engine._dump have been removed. These were not documented and are trivially replaced with calls to typedef._load and typedef._dump instead.

• (internal) The dumped attr for Conditions is no longer needed since there's no need to dump objects except at render time.

2.4.1 - 2019-10-11

Bug fix. Thanks to @wilfre in PR #141!

[Fixed]

• bloop.stream.shard.py::unpack_shards no longer raises when a Shard in the DescribeStream has a ParentId that is not also available in the DescribeStream response (the parent shard has been deleted). Previously the code would raise while trying to link the two shard objects in memory. Now, the shard will have a ParentId of None.

2.4.0 - 2019-06-13

The atomic= keyword for Engine.save and Engine.delete is deprecated and will be removed in 3.0. In 2.4 your code will continue to work but will raise DeprecationWarning when you specify a value for atomic=.

The Type._dump function return value is changing to Union[Any, bloop.Action] in 2.4 to prepare for the change in 3.0 to exclusively returning a bloop.Action. For built-in types and most custom types that only override dynamo_dump this is a no-op, but if you call Type._dump you can use bloop.actions.unwrap() on the result to get the inner value. If you have a custom Type._dump method it must return an action in 3.0. For ease of use you can use bloop.actions.wrap() which will specify either the SET or REMOVE action to match existing behavior. Here's an example of how you can quickly modify your code:

# current pre-2.4 method, continues to work until 3.0
def _dump(self, value, **kwargs):
    value = self.dynamo_dump(value, **kwargs)
    if value is None:
        return None
    return {self.backing_type: value}

# works in 2.4 and 3.0
from bloop import actions
def _dump(self, value, **kwargs):
    value = actions.unwrap(value)
    value = self.dynamo_dump(value, **kwargs)
    return actions.wrap(value)

Note that this is backwards compatible in 2.4: Type._dump will not change unless you opt to pass the new Action object to it.

[Added]

• SearchIterator.token provides a way to start a new Query or Scan from a previous query/scan's state. See Issue #132.

• SearchIterator.move_to takes a token to update the search state. Count/ScannedCount state are lost when moving to a token.

• Engine.delete and Engine.save take an optional argument sync= which can be used to update objects with the old or new values from DynamoDB after saving or deleting. See the Return Values section of the User Guide and Issue #137.

• bloop.actions expose a way to manipulate atomic counters and sets. See the Atomic Counters section of the User Guide and Issue #136.

[Changed]

• The atomic= keyword for Engine.save and Engine.delete emits DeprecationWarning and will be removed in 3.0.

• Type._dump will return a bloop.action.Action object if one is passed in, in preparation for the change in 3.0.

2.3.3 - 2019-01-27

Engine.bind is much faster for multi-model tables. See Issue #130.

[Changed]

• (internal) SessionWrapper caches DescribeTable responses. You can clear these with SessionWrapper.clear_cache; mutating calls such as .enable_ttl will invalidate the cached description.

• (internal) Each Engine.bind will call CreateTable at most once per table. Subsequent calls to bind will call CreateTable again.

2.3.2 - 2019-01-27

Minor bug fix.

[Fixed]

• (internal) bloop.conditions.iter_columns no longer yields None on Condition() (or any other condition whose .column attribute is None).

2.3.0 - 2019-01-24

This release adds support for Transactions and On-Demand Billing. Transactions can include changes across tables, and provide ACID guarantees at a 2x throughput cost and a limit of 10 items per transaction. See the User Guide for details.

with engine.transaction() as tx:
    tx.save(user, tweet)
    tx.delete(event, task)
    tx.check(meta, condition=Metadata.worker_id == current_worker)

[Added]

• Engine.transaction(mode="w") returns a transaction object which can be used directly or as a context manager. By default this creates a WriteTransaction, but you can pass mode="r" to create a read transaction.

• WriteTransaction and ReadTransaction can be prepared for committing with .prepare() which returns a PreparedTransaction which can be committed with .commit() some number of times. These calls are usually handled automatically when using the read/write transaction as a context manager:

  # manual calls
  tx = engine.transaction()
  tx.save(user)
  p = tx.prepare()
  p.commit()
  
  # equivalent functionality
  with engine.transaction() as tx:
      tx.save(user)
  

• Meta supports On-Demand Billing:

  class MyModel(BaseModel):
      id = Column(String, hash_key=True)
      class Meta:
          billing = {"mode": "on_demand"}
  

• (internal) bloop.session.SessionWrapper.transaction_read and bloop.session.SessionWrapper.transaction_write can be used to call TransactGetItems and TransactWriteItems with fully serialized request objects. The write api requires a client request token to provide idempotency guards, but does not provide temporal bounds checks for those tokens.

[Changed]

• Engine.load now logs at INFO instead of WARNING when failing to load some objects.

• Meta.ttl["enabled"] will now be a literal True or False after binding the model, rather than the string "enabled" or "disabled".

• If Meta.encryption or Meta.backups is None or missing, it will now be set after binding the model.

• Meta and GSI read/write units are not validated if billing mode is "on_demand" since they will be 0 and the provided setting is ignored.

2.2.0 - 2018-08-30

[Added]

• DynamicList and DynamicMap types can store arbitrary values, although they will only be loaded as their primitive, direct mapping to DynamoDB backing types. For example:

  class MyModel(BaseModel):
      id = Column(String, hash_key=True)
      blob = Column(DynamicMap)
  i = MyModel(id="i")
  i.blob = {"foo": "bar", "inner": [True, {1, 2, 3}, b""]}
  

• Meta supports Continuous Backups for Point-In-Time Recovery:

  class MyModel(BaseModel):
      id = Column(String, hash_key=True)
      class Meta:
          backups = {"enabled": True}
  

• SearchIterator exposes an all() method which eagerly loads all results and returns a single list. Note that the query or scan is reset each time the method is called, discarding any previously buffered state.

[Changed]

• String and Binary types load None as "" and b"" respectively.

• Saving an empty String or Binary ("" or b"") will no longer throw a botocore exception, and will instead be treated as None. This brings behavior in line with the Set, List, and Map types.

2.1.0 - 2018-04-07

Added support for Server-Side Encryption. This uses an AWS-managed Customer Master Key (CMK) stored in KMS which is managed for free: "You are not charged for the following: AWS-managed CMKs, which are automatically created on your behalf when you first attempt to encrypt a resource in a supported AWS service."

[Added]

• Meta supports Server Side Encryption:

  class MyModel(BaseModel):
      id = Column(String, hash_key=True)
      class Meta:
          encryption = {"enabled": True}
  

2.0.1 - 2018-02-03

Fix a bug where the last records in a closed shard in a Stream were dropped. See Issue #87 and PR #112.

[Fixed]

• Stream no longer drops the last records from a closed Shard when moving to the child shard.

2.0.0 - 2017-11-27

2.0.0 introduces 4 significant new features:

• Model inheritance and mixins

• Table name templates: table_name_template="prod-{table_name}"

• TTL support: ttl = {"column": "not_after"}

• Column defaults:

  verified=Column(Boolean, default=False)
  not_after = Column(
      Timestamp,
      default=lambda: (
          datetime.datetime.now() +
          datetime.timedelta(days=30)
      )
  )
  

Python 3.6.0 is now the minimum required version, as Bloop takes advantage of __set_name__ and __init_subclass__ to avoid the need for a Metaclass.

A number of internal-only and rarely-used external methods have been removed, as the processes which required them have been simplified:

• Column.get, Column.set, Column.delete in favor of their descriptor protocol counterparts

• bloop.Type._register is no longer necessary before using a custom Type

• Index._bind is replaced by helpers bind_index and refresh_index. You should not need to call these.

• A number of overly-specific exceptions have been removed.

[Added]

• Engine takes an optional keyword-only arg "table_name_template" which takes either a string used to format each name, or a function which will be called with the model to get the table name of. This removes the need to connect to the before_create_table signal, which also could not handle multiple table names for the same model. With this change BaseModel.Meta.table_name will no longer be authoritative, and the engine must be consulted to find a given model's table name. An internal function Engine._compute_table_name is available, and the per-engine table names may be added to the model.Meta in the future. (see Issue #96)

• A new exception InvalidTemplate is raised when an Engine's table_name_template is a string but does not contain the required "{table_name}" formatting key.

• You can now specify a TTL (see Issue #87) on a model much like a Stream:

  class MyModel(BaseModel):
      class Meta:
          ttl = {
              "column": "expire_after"
          }
  
  
      id = Column(UUID, hash_key=True)
      expire_after = Column(Timestamp)
  

• A new type, Timestamp was added. This stores a datetime.datetime as a unix timestamp in whole seconds.

• Corresponding Timestamp types were added to the following extensions, mirroring the DateTime extension: bloop.ext.arrow.Timestamp, bloop.ext.delorean.Timestamp, and bloop.ext.pendulum.Timestamp.

• Column takes an optional kwarg default, either a single value or a no-arg function that returns a value. Defaults are applied only during BaseModel.__init__ and not when loading objects from a Query, Scan, or Stream. If your function returns bloop.util.missing, no default will be applied. (see PR #90, PR #105 for extensive discussion)

• (internal) A new abstract interface, bloop.models.IMeta was added to assist with code completion. This fully describes the contents of a BaseModel.Meta instance, and can safely be subclassed to provide hints to your editor:

  class MyModel(BaseModel):
      class Meta(bloop.models.IMeta):
          table_name = "my-table"
      ...
  

• (internal) bloop.session.SessionWrapper.enable_ttl can be used to enable a TTL on a table. This SHOULD NOT be called unless the table was just created by bloop.

• (internal) helpers for dynamic model inheritance have been added to the bloop.models package:

  • bloop.models.bind_column

  • bloop.models.bind_index

  • bloop.models.refresh_index

  • bloop.models.unbind

  Direct use is discouraged without a strong understanding of how binding and inheritance work within bloop.

[Changed]

• Python 3.6 is the minimum supported version.

• BaseModel no longer requires a Metaclass, which allows it to be used as a mixin to an existing class which may have a Metaclass.

• BaseModel.Meta.init no longer defaults to the model's __init__ method, and will instead use cls.__new__(cls) to obtain an instance of the model. You can still specify a custom initialization function:

  class MyModel(BaseModel):
      class Meta:
          @classmethod
          def init(_):
              instance = MyModel.__new__(MyModel)
              instance.created_from_init = True
      id = Column(...)
  

• Column and Index support the shallow copy method __copy__ to simplify inheritance with custom subclasses. You may override this to change how your subclasses are inherited.

• DateTime explicitly guards against tzinfo is None, since datetime.astimezone started silently allowing this in Python 3.6 -- you should not use a naive datetime for any reason.

• Column.model_name is now Column.name, and Index.model_name is now Index.name.

• Column(name=) is now Column(dynamo_name=) and Index(name=) is now Index(dynamo_name=)

• The exception InvalidModel is raised instead of InvalidIndex.

• The exception InvalidSearch is raised instead of the following: InvalidSearchMode, InvalidKeyCondition, InvalidFilterCondition, and InvalidProjection.

• (internal) bloop.session.SessionWrapper methods now require an explicit table name, which is not read from the model name. This exists to support different computed table names per engine. The following methods now require a table name: create_table, describe_table (new), validate_table, and enable_ttl (new).

[Removed]

• bloop no longer supports Python versions below 3.6.0

• bloop no longer depends on declare

• Column.get, Column.set, and Column.delete helpers have been removed in favor of using the Descriptor protocol methods directly: Column.__get__, Column.__set__, and Column.__delete__.

• bloop.Type no longer exposes a _register method; there is no need to register types before using them, and you can remove the call entirely.

• Column.model_name, Index.model_name, and the kwargs Column(name=), Index(name=) (see above)

• The exception InvalidIndex has been removed.

• The exception InvalidComparisonOperator was unused and has been removed.

• The exception UnboundModel is no longer raised during Engine.bind and has been removed.

• The exceptions InvalidSearchMode, InvalidKeyCondition, InvalidFilterCondition, and InvalidProjection have been removed.

• (internal) Index._bind has been replaced with the more complete solutions in bloop.models.bind_column and bloop.models.bind_index.

1.3.0 - 2017-10-08

This release is exclusively to prepare users for the name/model_name/dynamo_name changes coming in 2.0; your 1.2.0 code will continue to work as usual but will raise DeprecationWarning when accessing model_name on a Column or Index, or when specifying the name= kwarg in the __init__ method of Column, GlobalSecondaryIndex, or LocalSecondaryIndex.

Previously it was unclear if Column.model_name was the name of this column in its model, or the name of the model it is attached to (eg. a shortcut for Column.model.__name__). Additionally the name= kwarg actually mapped to the object's .dynamo_name value, which was not obvious.

Now the Column.name attribute will hold the name of the column in its model, while Column.dynamo_name will hold the name used in DynamoDB, and is passed during initialization as dynamo_name=. Accessing model_name or passing name= during __init__ will raise deprecation warnings, and bloop 2.0.0 will remove the deprecated properties and ignore the deprecated kwargs.

[Added]

• Column.name is the new home of the Column.model_name attribute. The same is true for Index, GlobalSecondaryIndex, and LocalSecondaryIndex.

• The __init__ method of Column, Index, GlobalSecondaryIndex, and LocalSecondaryIndex now takes dynamo_name= in place of name=.

[Changed]

• Accessing Column.model_name raises DeprecationWarning, and the same for Index/GSI/LSI.

• Providing Column(name=) raises DeprecationWarning, and the same for Index/GSI/LSI.

1.2.0 - 2017-09-11

[Changed]

• When a Model's Meta does not explicitly set read_units and write_units, it will only default to 1/1 if the table does not exist and needs to be created. If the table already exists, any throughput will be considered valid. This will still ensure new tables have 1/1 iops as a default, but won't fail if an existing table has more than one of either.

  There is no behavior change for explicit integer values of read_units and write_units: if the table does not exist it will be created with those values, and if it does exist then validation will fail if the actual values differ from the modeled values.

  An explicit None for either read_units or write_units is equivalent to omitting the value, but allows for a more explicit declaration in the model.

  Because this is a relaxing of a default only within the context of validation (creation has the same semantics) the only users that should be impacted are those that do not declare read_units and write_units and rely on the built-in validation failing to match on values != 1. Users that rely on the validation to succeed on tables with values of 1 will see no change in behavior. This fits within the extended criteria of a minor release since there is a viable and obvious workaround for the current behavior (declare 1/1 and ensure failure on other values).

• When a Query or Scan has projection type "count", accessing the count or scanned properties will immediately execute and exhaust the iterator to provide the count or scanned count. This simplifies the previous workaround of calling next(query, None) before using query.count.

[Fixed]

• Fixed a bug where a Query or Scan with projection "count" would always raise KeyError (see Issue #95)

• Fixed a bug where resetting a Query or Scan would cause __next__ to raise botocore.exceptions.ParamValidationError (see Issue #95)

1.1.0 - 2017-04-26

[Added]

• Engine.bind takes optional kwarg skip_table_setup to skip CreateTable and DescribeTable calls (see Issue #83)

• Index validates against a superset of the projection (see Issue #71)

1.0.3 - 2017-03-05

Bug fix.

[Fixed]

• Stream orders records on the integer of SequenceNumber, not the lexicographical sorting of its string representation. This is an annoying bug, because as documented we should be using lexicographical sorting on the opaque string. However, without leading 0s that sort fails, and we must assume the string represents an integer to sort on. Particularly annoying, tomorrow the SequenceNumber could start with non-numeric characters and still conform to the spec, but the sorting-as-int assumption breaks. However, we can't properly sort without making that assumption.

1.0.2 - 2017-03-05

Minor bug fix.

[Fixed]

• extension types in ext.arrow, ext.delorean, and ext.pendulum now load and dump None correctly.

1.0.1 - 2017-03-04

Bug fixes.

[Changed]

• The arrow, delorean, and pendulum extensions now have a default timezone of "utc" instead of datetime.timezone.utc. There are open issues for both projects to verify if that is the expected behavior.

[Fixed]

• DynamoDBStreams return a Timestamp for each record's ApproximateCreationDateTime, which botocore is translating into a real datetime.datetime object. Previously, the record parser assumed an int was used. While this fix is a breaking change for an internal API, this bug broke the Stream iterator interface entirely, which means no one could have been using it anyway.

1.0.0 - 2016-11-16

1.0.0 is the culmination of just under a year of redesigns, bug fixes, and new features. Over 550 commits, more than 60 issues closed, over 1200 new unit tests. At an extremely high level:

• The query and scan interfaces have been polished and simplified. Extraneous methods and configuration settings have been cut out, while ambiguous properties and methods have been merged into a single call.

• A new, simple API exposes DynamoDBStreams with just a few methods; no need to manage individual shards, maintain shard hierarchies and open/closed polling. I believe this is a first since the Kinesis Adapter and KCL, although they serve different purposes. When a single worker can keep up with a model's stream, Bloop's interface is immensely easier to use.

• Engine's methods are more consistent with each other and across the code base, and all of the configuration settings have been made redundant. This removes the need for EngineView and its associated temporary config changes.

• Blinker-powered signals make it easy to plug in additional logic when certain events occur: before a table is created; after a model is validated; whenever an object is modified.

• Types have been pared down while their flexibility has increased significantly. It's possible to create a type that loads another object as a column's value, using the engine and context passed into the load and dump functions. Be careful with this; transactions on top of DynamoDB are very hard to get right.

See the Migration Guide above for specific examples of breaking changes and how to fix them, or the User Guide for a tour of the new Bloop. Lastly, the Public and Internal API References are finally available and should cover everything you need to extend or replace whole subsystems in Bloop (if not, please open an issue).

[Added]

• bloop.signals exposes Blinker signals which can be used to monitor object changes, when instances are loaded from a query, before models are bound, etc.

  • before_create_table

  • object_loaded

  • object_saved

  • object_deleted

  • object_modified

  • model_bound

  • model_created

  • model_validated

• Engine.stream can be used to iterate over all records in a stream, with a total ordering over approximate record creation time. Use engine.stream(model, "trim_horizon") to get started. See the User Guide for details.

• New exceptions RecordsExpired and ShardIteratorExpired for errors in stream state

• New exceptions Invalid* for bad input subclass BloopException and ValueError

• DateTime types for the three most common date time libraries:

  • bloop.ext.arrow.DateTime

  • bloop.ext.delorean.DateTime

  • bloop.ext.pendulum.DateTime

• model.Meta has a new optional attribute stream which can be used to enable a stream on the model's table.

• model.Meta exposes the same projection attribute as Index so that (index or model.Meta).projection can be used interchangeably

• New Stream class exposes DynamoDBStreams API as a single iterable with powerful seek/jump options, and simple json-friendly tokens for pausing and resuming iteration.

• Over 1200 unit tests added

• Initial integration tests added

• (internal) bloop.conditions.ReferenceTracker handles building #n0, :v1, and associated values. Use any_ref to build a reference to a name/path/value, and pop_refs when backtracking (eg. when a value is actually another column, or when correcting a partially valid condition)

• (internal) bloop.conditions.render is the preferred entry point for rendering, and handles all permutations of conditions, filters, projections. Use over ConditionRenderer unless you need very specific control over rendering sequencing.

• (internal) bloop.session.SessionWrapper exposes DynamoDBStreams operations in addition to previous bloop.Client wrappers around DynamoDB client

• (internal) New supporting classes streams.buffer.RecordBuffer, streams.shard.Shard, and streams.coordinator.Coordinator to encapsulate the hell^Wjoy that is working with DynamoDBStreams

• (internal) New class util.Sentinel for placeholder values like missing and last_token that provide clearer docstrings, instead of showing func(..., default=object<0x...>) these will show func(..., default=Sentinel<[Missing]>)

[Changed]

• bloop.Column emits object_modified on __set__ and __del__

• Conditions now check if they can be used with a column's typedef and raise InvalidCondition when they can't. For example, contains can't be used on Number, nor > on Set(String)

• bloop.Engine no longer takes an optional bloop.Client but instead optional dynamodb and dynamodbstreams clients (usually created from boto3.client("dynamodb") etc.)

• Engine no longer takes **config -- its settings have been dispersed to their local touch points

  • atomic is a parameter of save and delete and defaults to False

  • consistent is a parameter of load, query, scan and defaults to False

  • prefetch has no equivalent, and is baked into the new Query/Scan iterator logic

  • strict is a parameter of a LocalSecondaryIndex, defaults to True

• Engine no longer has a context to create temporary views with different configuration

• Engine.bind is no longer by keyword arg only: engine.bind(MyBase) is acceptable in addition to engine.bind(base=MyBase)

• Engine.bind emits new signals before_create_table, model_validated, and model_bound

• Engine.delete and Engine.save take *objs instead of objs to easily save/delete small multiples of objects (engine.save(user, tweet) instead of engine.save([user, tweet]))

• Engine guards against loading, saving, querying, etc against abstract models

• Engine.load raises MissingObjects instead of NotModified (exception rename)

• Engine.scan and Engine.query take all query and scan arguments immediately, instead of using the builder pattern. For example, engine.scan(model).filter(Model.x==3) has become engine.scan(model, filter=Model.x==3).

• bloop.exceptions.NotModified renamed to bloop.exceptions.MissingObjects

• Any code that raised AbstractModelException now raises UnboundModel

• bloop.types.DateTime is now backed by datetime.datetime instead of arrow. Only supports UTC now, no local timezone. Use the bloop.ext.arrow.DateTime class to continue using arrow.

• The query and scan interfaces have been entirely refactored: count, consistent, ascending and other properties are part of the Engine.query(...) parameters. all() is no longer needed, as Engine.scan and .query immediately return an iterable object. There is no prefetch setting, or limit.

• The complete property for Query and Scan have been replaced with exhausted, to be consistent with the Stream module

• The query and scan iterator no longer cache results

• The projection parameter is now required for GlobalSecondaryIndex and LocalSecondaryIndex

• Calling Index.__set__ or Index.__del__ will raise AttributeError. For example, some_user.by_email = 3 raises if User.by_email is a GSI

• bloop.Number replaces bloop.Float and takes an optional decimal.Context for converting numbers. For a less strict, lossy Float type see the Patterns section of the User Guide

• bloop.String.dynamo_dump no longer calls str() on the value, which was hiding bugs where a non-string object was passed (eg. some_user.name = object() would save with a name of <object <0x...>)

• bloop.DateTime is now backed by datetime.datetime and only knows UTC in a fixed format. Adapters for arrow, delorean, and pendulum are available in bloop.ext

• bloop.DateTime does not support naive datetimes; they must always have a tzinfo

• docs:

  • use RTD theme

  • rewritten three times

  • now includes public and internal api references

• (internal) Path lookups on Column (eg. User.profile["name"]["last"]) use simpler proxies

• (internal) Proxy behavior split out from Column's base class bloop.conditions.ComparisonMixin for a cleaner namespace

• (internal) bloop.conditions.ConditionRenderer rewritten, uses a new bloop.conditions.ReferenceTracker with a much clearer api

• (internal) ConditionRenderer can backtrack references and handles columns as values (eg. User.name.in_([User.email, "literal"]))

• (internal) _MultiCondition logic rolled into bloop.conditions.BaseCondition, AndCondition and OrCondition no longer have intermediate base class

• (internal) AttributeExists logic rolled into bloop.conditions.ComparisonCondition

• (internal) bloop.tracking rolled into bloop.conditions and is hooked into the object_* signals. Methods are no longer called directly (eg. no need for tracking.sync(some_obj, engine))

• (internal) update condition is built from a set of columns, not a dict of updates to apply

• (internal) bloop.conditions.BaseCondition is a more comprehensive base class, and handles all manner of out-of-order merges (and(x, y) vs and(y, x) where x is an and condition and y is not)

• (internal) almost all *Condition classes simply implement __repr__ and render; BaseCondition takes care of everything else

• (internal) bloop.Client became bloop.session.SessionWrapper

• (internal) Engine._dump takes an optional context, **kwargs, matching the signature of Engine._load

• (internal) BaseModel no longer implements __hash__, __eq__, or __ne__ but ModelMetaclass will always ensure a __hash__ function when the subclass is created

• (internal) Filter and FilterIterator rewritten entirely in the bloop.search module across multiple classes

[Removed]

• AbstractModelException has been rolled into UnboundModel

• The all() method has been removed from the query and scan iterator interface. Simply iterate with next(query) or for result in query:

• Query.results and Scan.results have been removed and results are no longer cached. You can begin the search again with query.reset()

• The new_base() function has been removed in favor of subclassing BaseModel directly

• bloop.Float has been replaced by bloop.Number

• (internal) bloop.engine.LoadManager logic was rolled into bloop.engine.load(...)

• EngineView has been removed since engines no longer have a baseline config and don't need a context to temporarily modify it

• (internal) Engine._update has been removed in favor of util.unpack_from_dynamodb

• (internal) Engine._instance has been removed in favor of directly creating instances from model.Meta.init() in unpack_from_dynamodb

[Fixed]

• Column.contains(value) now renders value with the column typedef's inner type. Previously, the container type was used, so Data.some_list.contains("foo")) would render as (contains(some_list, ["f", "o", "o"])) instead of (contains(some_list, "foo"))

• Set renders correct wire format -- previously, it incorrectly sent {"SS": [{"S": "h"}, {"S": "i"}]} instead of the correct {"SS": ["h", "i"]}

• (internal) Set and List expose an inner_typedef for conditions to force rendering of inner values (currently only used by ContainsCondition)

0.9.13 - 2016-10-31

[Fixed]

• Set was rendering an invalid wire format, and now renders the correct "SS", "NS", or "BS" values.

• Set and List were rendering contains conditions incorrectly, by trying to dump each value in the value passed to contains. For example, MyModel.strings.contains("foo") would render contains(#n0, :v1) where :v1 was {"SS": [{"S": "f"}, {"S": "o"}, {"S": "o"}]}. Now, non-iterable values are rendered singularly, so :v1 would be {"S": "foo"}. This is a temporary fix, and only works for simple cases. For example, List(List(String)) will still break when performing a contains check. This is fixed correctly in 1.0.0 and you should migrate as soon as possible.

0.9.12 - 2016-06-13

[Added]

• model.Meta now exposes gsis and lsis, in addition to the existing indexes. This simplifies code that needs to iterate over each type of index and not all indexes.

[Removed]

• engine_for_profile was no longer necessary, since the client instances could simply be created with a given profile.

0.9.11 - 2016-06-12

[Changed]

• bloop.Client now takes boto_client, which should be an instance of boto3.client("dynamodb") instead of a boto3.session.Session. This lets you specify endpoints and other configuration only exposed during the client creation process.

• Engine no longer uses "session" from the config, and instead takes a client param which should be an instance of bloop.Client. bloop.Client will be going away in 1.0.0 and Engine will simply take the boto3 clients directly.

0.9.10 - 2016-06-07

[Added]

• New exception AbstractModelException is raised when attempting to perform an operation which requires a table, on an abstract model. Raised by all Engine functions as well as bloop.Client operations.

[Changed]

• Engine operations raise AbstractModelException when attempting to perform operations on abstract models.

• Previously, models were considered non-abstract if model.Meta.abstract was False, or there was no value. Now, ModelMetaclass will explicitly set abstract to False so that model.Meta.abstract can be used everywhere, instead of getattr(model.Meta, "abstract", False).

0.9.9 - 2016-06-06

[Added]

• Column has a new attribute model, the model it is bound to. This is set during the model's creation by the ModelMetaclass.

[Changed]

• Engine.bind will now skip intermediate models that are abstract. This makes it easier to pass abstract models, or models whose subclasses may be abstract (and have non-abstract grandchildren).

0.9.8 - 2016-06-05

(no public changes)

0.9.7 - 2016-06-05

[Changed]

• Conditions implement __eq__ for checking if two conditions will evaluate the same. For example:

  >>> large = Blob.size > 1024**2
  >>> small = Blob.size < 1024**2
  >>> large == small
  False
  >>> also_large = Blob.size > 1024**2
  >>> large == also_large
  True
  >>> large is also_large
  False
  

0.9.6 - 2016-06-04

0.9.6 is the first significant change to how Bloop binds models, engines, and tables. There are a few breaking changes, although they should be easy to update.

Where you previously created a model from the Engine's model:

from bloop import Engine

engine = Engine()

class MyModel(engine.model):
    ...

You'll now create a base without any relation to an engine, and then bind it to any engines you want:

from bloop import Engine, new_base

BaseModel = new_base()

class MyModel(BaseModel):
    ...

engine = Engine()
engine.bind(base=MyModel)  # or base=BaseModel

[Added]

• A new function engine_for_profile takes a profile name for the config file and creates an appropriate session. This is a temporary utility, since Engine will eventually take instances of dynamodb and dynamodbstreams clients. This will be going away in 1.0.0.

• A new base exception BloopException which can be used to catch anything thrown by Bloop.

• A new function new_base() creates an abstract base for models. This replaces Engine.model now that multiple engines can bind the same model. This will be going away in 1.0.0 which will provide a BaseModel class.

[Changed]

• The session parameter to Engine is now part of the config kwargs. The underlying bloop.Client is no longer created in Engine.__init__, which provides an opportunity to swap out the client entirely before the first Engine.bind call. The semantics of session and client are unchanged.

• Engine._load, Engine._dump, and all Type signatures now pass an engine explicitly through the context parameter. This was mentioned in 0.9.2 and context is now required.

• Engine.bind now binds the given class and all subclasses. This simplifies most workflows, since you can now create a base with MyBase = new_base() and then bind every model you create with engine.bind(base=MyBase).

• All exceptions now subclass a new base exception BloopException instead of Exception.

• Vector types Set, List, Map, and TypedMap accept a typedef of None so they can raise a more helpful error message. This will be reverted in 1.0.0 and will once again be a required parameter.

[Removed]

• Engine no longer has model, unbound_models, or models attributes. Engine.model has been replaced by the new_base() function, and models are bound directly to the underlying type engine without tracking on the Engine instance itself.

• EngineView dropped the corresponding attributes above.

0.9.5 - 2016-06-01

[Changed]

• EngineView attributes are now properties, and point to the underlying engine's attributes; this includes client, model, type_engine, and unbound_models. This fixed an issue when using with engine.context(...) as view: to perform operations on models bound to the engine but not the engine view. EngineView will be going away in 1.0.0.

0.9.4 - 2015-12-31

[Added]

• Engine functions now take optional config parameters to override the engine's config. You should update your code to use these values instead of engine.config, since engine.config is going away in 1.0.0. Engine.delete and Engine.save expose the atomic parameter, while Engine.load exposes consistent.

• Added the TypedMap class, which provides dict mapping for a single typedef over any number of keys. This differs from Map, which must know all keys ahead of time and can use different types. TypedMap only supports a single type, but can have arbitrary keys. This will be going away in 1.0.0.

0.9.2 - 2015-12-11

[Changed]

• Type functions _load, _dump, dynamo_load, dynamo_dump now take an optional keyword-only arg context. This dict will become required in 0.9.6, and contains the engine instance that should be used for recursive types. If your type currently uses cls.Meta.bloop_engine, you should start using context["engine"] in the next release. The bloop_engine attribute is being removed, since models will be able to bind to multiple engines.

0.9.1 - 2015-12-07

(no public changes)

0.9.0 - 2015-12-07