borgcube.core package

borgcube.core.hookspec module

borgcube.core.hookspec.borgcube_startup(process)

Called after application is configured and all plugins were discovered.

process identifies the process and may be None. Otherwise it can be one of:

• “borgcubed”
• “proxy”
• “manage” (this may include running the web frontend)

borgcube.core.hookspec.borgcube_job_blocked(job, blocking_jobs)

Called to sort out whether job is blocked by any blocking_jobs.

blocking_jobs is a list of jobs that are aimed at the same repository as job and whose state is not in Job.State.STABLE.

Remove any false positives from blocking_jobs that might be there because of special snowflake semantics with your jobs.

If any blocking_jobs remain, job is considered blocked.

borgcube.core.hookspec.borgcube_job_pre_state_update(job, current_state, target_state)

Called before the state of job is updated from current_state (BackupJob.State) to target_state (ditto).

borgcube.core.hookspec.borgcube_job_post_state_update(job, prior_state, current_state)

Called after the state of job was updated from prior_state (BackupJob.State) to current_state.

job is already saved, but the database transaction has not yet completed.

borgcube.core.hookspec.borgcube_job_post_force_state(job, forced_state)

Called after the state of job was forced to forced_state (BackupJob.State).

borgcube.core.hookspec.borgcube_job_failure_cause(job, kind, kwargs)

Called after the failure cause (defined by kind (str) and it’s kwargs) is set for job.

borgcube.core.hookspec.borgcube_job_created(job)

Called when a job was created, but before it will be executed.

borgcube.core.hookspec.borgcube_job_pre_delete(job)

Called before job will be deleted.

borgcube.core.hookspec.borgcube_archive_added(archive)

Called after archive was added (core.models.Archive).

borgcube.core.hookspec.borgcube_archive_pre_delete(archive)

Called before archive (core.models.Archive) will be deleted.

borgcube.core.models module

class borgcube.core.models.NumberTree

Bases: BTrees.LOBTree.LOBTree

insert(key, value) → 0 or 1

Add an item if the key is not already used. Return 1 if the item was added, or 0 otherwise.

reversed()

Yield values in descending (high to low) key order..

class borgcube.core.models.PersistentDefaultDict(*args, factory)

Bases: persistent.mapping.PersistentMapping

class borgcube.core.models.StringObjectID

Bases: object

oid

class borgcube.core.models.Updateable

Bases: object

class borgcube.core.models.Volatility

Bases: object

borgcube.core.models.evolve(from_version=1, to_version=None)

class borgcube.core.models.Evolvable

Bases: persistent.Persistent, borgcube.core.models.StringObjectID, borgcube.core.models.Updateable

Main class for persistent data in BorgCube.

This provides a means for online data migration (“evolution”). For this both objects and classes are versioned through the version attribute. When an object is written to the database the current version is stored along with it (via __getstate__), while retrieving an object may evolve it to the current version of the class.

This is best illustrated with a simple example:

class Dog(Evolvable):
    def __init__(self):
        pass

Fair enough. We acquire some dogs, but notice we forgot a few things, which we want to add:

class Dog(Evolvable):
    def __init__(self, good_boy, food_bowl):
        self.good_boy = good_boy
        self.food_bowl = food_bowl

New code will expect these attributes to be available, so it would fail with older dogs. We fix this through evolution:

class Dog(Evolvable):
    version = 2

    @evolve(from_version=1, to_version=2)
    def added_important_things(self):
        self.good_boy = True
        self.food_bowl = FoodBowl.find_free_bowl_or_create_one()

    def __init__(self, good_boy, food_bowl):
        self.good_boy = good_boy
        self.food_bowl = food_bowl

If now an old dog (version=1) is loaded, the system will notice that and run added_important_things, which will perform the changes needed to match version=2 dogs.

Note that this is subject to the regular transaction rules, so when, after loading old objects, no commit happens, the in-database data isn’t updated – the evolution would happen again next time they are loaded.

You can also provide shortcuts (or leaps), if there is a better upgrade path between certain versions, like so:

class Dog(Evolvable):
    version = 7

    @evolve(from_version=2, to_version=7):
    def direct_upgrade_2to7(self):
        ...

    @evolve(from_version=2, to_version=3):
    def upgrade_2to3(self):
        ...

Note that evolution isn’t a SAT-solver, but works in a short-circuit way; in each step the largest leap is selected. Hidden paths are not considered.

version = 1

class borgcube.core.models.DataRoot

Bases: borgcube.core.models.Evolvable

The DataRoot has the following attributes:

Variables:

• repositories – a PersistentList of Repository instances.
• archives – an OOBTree mapping hex archive IDs to Archive instances.
• clients – an OOBTree mapping host names to Client instances.
• jobs – an OOBTree mapping TODO to Job instances.
• jobs_by_state – an OOBTree mapping job states to trees of Job instances.
• schedules – a PersistentList of Schedule instances.
• ext – a PersistentDict of extension data (see plugin_data, do not use directly).

version = 6

add_ext_dict()

ensure_base_job_states()

defaultdict()

numbered_jobs()

add_triggers()

plugin_data(factory)

Return an instance generated (at some point in time) by factory.

This should be used for storing plugin data, eg.:

class AwesomePluginData(Evolvable):
    name = 'awesome-plugin'

    def __init__(self):
        self.some_data = OOBTree()

...

def show_some_data(request):
    # You might want to just put this in a separate helper (below)
    plugdat = data_root().plugin_data(AwesomePluginData)
    return TemplateResponse(...)

# A sample helper
def plugin_root():
    return data_root().plugin_data(AwesomePluginData)

A good name would be the entrypoint name of your plugin, or it’s root module/package name.

The name attribute on factory is not mandatory. If it is not present the qualified class name is used instead (eg. awesomeplugin.data.AwesomePluginData)

class borgcube.core.models.Repository(name, url, description='', repository_id='', remote_borg='borg')

Bases: borgcube.core.models.Evolvable

version = 2

add_job_configs()

location

latest_job()

static oid_get(oid)

class Form(data=None, files=None, auto_id='id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, field_order=None, use_required_attribute=None)

Bases: django.forms.forms.Form

name = django.forms.CharField

description = django.forms.CharField

url = django.forms.CharField

repository_id = django.forms.CharField

remote_borg = django.forms.CharField

class ChoiceField(**kwargs)

Bases: django.forms.fields.ChoiceField

static get_choices()

clean(value)

Validates the given value and returns its “cleaned” value as an appropriate Python object.

Raises ValidationError for any errors.

prepare_value(value)

class borgcube.core.models.Archive(id, repository, name, client=None, job=None, comment='', nfiles=0, original_size=0, compressed_size=0, deduplicated_size=0, duration=datetime.timedelta(0), timestamp=None, timestamp_end=None)

Bases: borgcube.core.models.Evolvable

version = 2

add_timestamps()

ts

delete(manifest, stats, cache)

class borgcube.core.models.RshClientConnection(remote, rsh='ssh', rsh_options=None, ssh_identity_file=None, remote_borg='borg', remote_cache_dir=None)

Bases: borgcube.core.models.Evolvable

remote = ''

rsh = 'ssh'

rsh_options = None

ssh_identity_file = None

remote_borg = 'borg'

remote_cache_dir = None

class Form(data=None, files=None, auto_id='id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, field_order=None, use_required_attribute=None)

Bases: django.forms.forms.Form

remote = django.forms.CharField

rsh = django.forms.CharField

rsh_options = django.forms.CharField

ssh_identity_file = django.forms.CharField

remote_borg = django.forms.CharField

remote_cache_dir = django.forms.CharField

prefix = 'connection'

class borgcube.core.models.Client(hostname, description='', connection=None)

Bases: borgcube.core.models.Evolvable

version = 2

add_job_configs()

latest_job()

class Form(data=None, files=None, auto_id='id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, field_order=None, use_required_attribute=None)

Bases: django.forms.forms.Form

hostname = django.forms.CharField

description = django.forms.CharField

class borgcube.core.models.s

Bases: str

class borgcube.core.models.JobExecutor

Bases: object

name = 'job-executor'

classmethod can_run(job)

classmethod prefork(job)

classmethod run(job)

class borgcube.core.models.Job(repository=None)

Bases: borgcube.core.models.Evolvable

Core job model.

A job is some kind of task that is run by borgcubed. Normally these are concerned with a repository, but if that’s not the case - the field is nullable.

Steps to implement a Job class:

1. Derive from this
2. Define additional states, if necessary, by deriving the State class in your model from Job.State
3. borgcubed needs to know how to run the job, therefore set executor to your JobExecutor subclass.
4. Relevant hooks: borgcube_job_blocked, borgcubed_job_exit.

short_name = 'job'

executor

alias of JobExecutor

class State

Bases: object

job_created = 'job_created'

done = 'done'

failed = 'failed'

cancelled = 'cancelled'

STABLE = {'cancelled', 'done', 'failed', 'job_created'}

classmethod verbose_name(name)

duration

failed

done

stable

update_state(previous, to)

force_state(state)

set_failure_cause(kind, **kwargs)

log_path()

delete(using=None, keep_parents=False)

class borgcube.core.models.TriggerID(trigger, enabled=True, access=(), comment='')

Bases: borgcube.core.models.Evolvable

Defined access contexts are:

• anonymous-web
• local-cli

run(access_context)

class borgcube.core.models.Trigger(target)

Bases: borgcube.core.models.Evolvable

class borgcube.core.models.Schedule(name, recurrence, recurrence_enabled=True, description='')

Bases: borgcube.core.models.Evolvable

version = 4

make_dtstart_implicit()

add_recurrence_enabled()

add_trigger()

run_from_trigger()

class Form(data, *args, **kwargs)

Bases: django.forms.forms.Form

name = django.forms.CharField

description = django.forms.CharField

recurrence_enabled = django.forms.BooleanField

recurrence_start = django.forms.DateTimeField

recurrence = recurrence.forms.RecurrenceField

clean()

Hook for doing any extra form-wide cleaning after Field.clean() has been called on every field. Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field named ‘__all__’.

class borgcube.core.models.DottedPath

Bases: object

classmethod dotted_path()

class borgcube.core.models.ScheduledAction(schedule)

Bases: borgcube.core.models.Evolvable, borgcube.core.models.DottedPath

Implement this to add schedulable actions to the scheduler.

Make sure that your implementation is imported, since these are implicitly discovered subclasses.

name = ''

class Form(data=None, files=None, auto_id='id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, field_order=None, use_required_attribute=None)

Bases: django.forms.forms.Form

The form that should be presented for adding/modifying this action.

This can be a ModelForm or modify the DB; the transaction is managed for you, and no additional transaction management should be needed.

The usual form rules apply, however, note that .cleaned_data must be JSON serializable if .is_valid() returns true. This data will be used for instanciating the action class (py_args).

.save() will be called with no arguments regardless of type, if it exists.

classmethod form(*args, **kwargs)

execute(apiserver)

classmethod get_class(dotted_path)