Migration Examples
As a supplement to the migrations documentation, this page gives some code examples and various common scenarios, showing how Djangae migrations may be used in practice.
Example Migration File
This migration file would live alongside your auto-generated Django migration files inside the migrations
folder in your Django app. You should name the file just like you would with a normal Django migration file, e.g. 0004_add_favourite_colour_field_data.py
.
from djangae.db.migrations.operations import AddFieldData
from djangae.fields import CharField
from django.db import migrations
class Migration(migrations.Migration):
""" Populates the Person model with the default value for the new `favourite_colour` field. """
operations = [
AddFieldData(
"myapp.Person",
"favourite_colour",
CharField(default="blue")
)
]
Example Migration Scenarios
One of the great advantages of a schemaless database is that you can incrementally modify the data structure, even on extremely large tables, and even while the application is in use. However, this departure from the "Turn it off, run a migration, turn it back on again" approach means that planning data migrations often requires a bit more thought.
Below are some examples of how one might approach different migration scenarios.
Add Field
Steps
- Run the migraton containing the normal
django...AddField
operation - Deploy new model code.
- Run a migration containing the
djangae...AddFieldData
operation to populate the new field (optional). - Deploy code which queries on the new field (optional).
Explanation
If you don't need to query on the default value of the new field, then all you need to do is add the field to your model (step 1) and ensure that the field either has a default
, has null=True
, or is a CharField
or TextField
(which effectively have a default of ""
).
As instances are loaded from the DB and re-saved as part of the general running of your application, they will be re-saved with the default value.
You can skip steps 3 and 4.
If however, you need to be able to filter on the default value of the new field in your queries, then you will need to run step 3 in order to populate the value into the DB (and thus into the Datastore indexes).
It is recommended that you deploy the new model code before runing the AddFieldData
operation.
This is because if you populate the existing objects in the Datastore with the default field value before adding the field to the model, then if your application creates any new model instances while the AddFieldData
process is running then they may not get caught by that process and you may therefore end up with objects which do not have the default value set.
But if, as recommended, you add the field to your model first, then any new objects created will get the default value, and the existing objects will be populated by the AddFieldData
process, meaning that no objects are missed.
Remove Field
Steps
- Run the migration containing the normal
django...RemoveField
operation. - Deploy new model code.
- Run a migration containing a
djangae...RemoveFieldData
operation (optional).
Explanation
If you remove a field from a model and you do not run the RemoveFieldData
operation, then any existing objects will simply keep the old field value in their underlying entities in the Datastore.
The only problem with this is that it takes up storage space in your Datastore which costs you money.
But note that running the RemoveFieldData
process will cause Datastore writes and use instance hours, which will also cost you money.
Whether you want to run the RemoveFieldData
task or not depends on how much data there is and how long you expect your application to live for, weighted with the various costs involved in storing or removing it.
Rename Field
Steps
- Run the migration containing the normal
django...AddField(new_field)
operation. - Create
save
method to ensure that any value which is saved to the old field is also saved to the new field. - Deploy new code.
- Run migrations containing the following operations (note that these are a mix of django and djangae operations, so each must be in a separate migration file):
djangae...CopyFieldData(old_field, new_Field)
.django...RemoveField(old_field)
.djangae...RemoveFieldData(old_field)
(optional).
Explanation
Renaming a field requires adding a new field, copying the data from the old field, and then removing the old field, but in doing so ensuring that any objects which are created or edited during that process have their (latest) values for the old field copied across.
Delete Model Data
Steps
- Run the migration containing the normal
django...AddField(new_field)
operation. - Deploy new code.
- Run a migration containing a
djangae... DeleteModelData
operation (optional).
Explanation
The DeleteModelData
operation deletes all the data related to a particular model (essentially DROP TABLE X
). There is no special process required here, just use this operation carefully!
Copy Data from One Model to Another
The CopyModelData
operation copies all the row data for a model into another. Both model classes must exist in your project for the operation to work. The exact steps that you use here will probably depend on what you're doing with those models.
Copy Data for a Model into a Different Namespace
The CopyModelDataToNamespace
operation will effectively copy an entire table (kind) from one Datastore namespace to another. The exact steps that you use here will probably depend on what you're doing with those 2 namespaces.
Custom Changes
For any other custom data manipulations, you can use the MapFunctionOnEntities
operation to call a custom function on each entities of a model class.